{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "# EnsembleVoteClassifier"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Implementation of a majority voting `EnsembleVoteClassifier` for classification."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> from mlxtend.classifier import EnsembleVoteClassifier"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Overview"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `EnsembleVoteClassifier` is a meta-classifier for combining similar or conceptually different machine learning classifiers for classification via majority or plurality voting. (For simplicity, we will refer to both majority and plurality voting as majority voting.)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "![](./EnsembleVoteClassifier_files/voting.png)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `EnsembleVoteClassifier` implements \"hard\" and \"soft\" voting. In hard voting, we predict the final class label as the class label that has been predicted most frequently by the classification models. In soft voting, we predict the class labels by averaging the class-probabilities (only recommended if the classifiers are well-calibrated).\n",
    "\n",
    "![](./EnsembleVoteClassifier_files/majority_voting.png)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Note**\n",
    "\n",
    "If you are interested in using the <code>EnsembleVoteClassifier</code>, please note that it is now also available through scikit learn (>0.17) as <a href=\"http://scikit-learn.org/stable/modules/generated/sklearn.ensemble.VotingClassifier.html\"><code>VotingClassifier</code></a>."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Majority Voting / Hard Voting"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Hard voting is the simplest case of majority voting. Here, we predict the class label $\\hat{y}$ via majority (plurality) voting of each classifier $C_j$:\n",
    "\n",
    "$$\\hat{y}=mode\\{C_1(\\mathbf{x}), C_2(\\mathbf{x}), ..., C_m(\\mathbf{x})\\}$$\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "\n",
    "Assuming that we combine three classifiers that classify a training sample as follows:\n",
    "\n",
    "- classifier 1 -> class 0\n",
    "- classifier 2 -> class 0\n",
    "- classifier 3 -> class 1\n",
    "\n",
    "$$\\hat{y}=mode\\{0, 0, 1\\} = 0$$\n",
    "\n",
    "Via majority vote, we would we would classify the sample as \"class 0.\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Weighted Majority Vote"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In addition to the simple majority vote (hard voting) as described in the previous section, we can compute a weighted majority vote by associating a weight $w_j$ with classifier $C_j$:\n",
    "\n",
    "$$\\hat{y} = \\arg \\max_i \\sum^{m}_{j=1} w_j \\chi_A \\big(C_j(\\mathbf{x})=i\\big),$$\n",
    "\n",
    "where $\\chi_A$ is the characteristic function $[C_j(\\mathbf{x}) = i \\; \\in A]$, and $A$ is the set of unique class labels. \n",
    "\n",
    "Continuing with the example from the previous section\n",
    "\n",
    "- classifier 1 -> class 0\n",
    "- classifier 2 -> class 0\n",
    "- classifier 3 -> class 1\n",
    "\n",
    "assigning the weights \\{0.2, 0.2, 0.6\\} would yield a prediction $\\hat{y} = 1$:\n",
    "\n",
    "$$\\arg \\max_i [0.2 \\times i_0 + 0.2 \\times i_0 + 0.6 \\times i_1] = 1$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Soft Voting"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In soft voting, we predict the class labels based on the predicted probabilities $p$ for classifier -- this approach is only recommended if the classifiers are well-calibrated.\n",
    "\n",
    "$$\\hat{y} = \\arg \\max_i \\sum^{m}_{j=1} w_j p_{ij},$$\n",
    "\n",
    "where $w_j$ is the weight that can be assigned to the $j$th classifier.\n",
    "\n",
    "Assuming the example in the previous section was a binary classification task with class labels $i \\in \\{0, 1\\}$, our ensemble could make the following prediction:\n",
    "\n",
    "- $C_1(\\mathbf{x}) \\rightarrow [0.9, 0.1]$\n",
    "- $C_2(\\mathbf{x}) \\rightarrow [0.8, 0.2]$\n",
    "- $C_3(\\mathbf{x}) \\rightarrow [0.4, 0.6]$\n",
    "\n",
    "Using uniform weights, we compute the average probabilities:\n",
    "\n",
    "$$p(i_0 \\mid \\mathbf{x}) = \\frac{0.9 + 0.8 + 0.4}{3} = 0.7 \\\\\\\\\n",
    "p(i_1 \\mid \\mathbf{x}) = \\frac{0.1 + 0.2 + 0.6}{3} = 0.3$$\n",
    "\n",
    "$$\\hat{y} = \\arg \\max_i \\big[p(i_0 \\mid \\mathbf{x}), p(i_1 \\mid \\mathbf{x}) \\big] = 0$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "However, assigning the weights \\{0.1, 0.1, 0.8\\} would yield a prediction $\\hat{y} = 1$:\n",
    "    \n",
    "$$p(i_0 \\mid \\mathbf{x}) = {0.1 \\times 0.9 + 0.1 \\times 0.8 + 0.8 \\times  0.4} = 0.49 \\\\\\\\\n",
    "p(i_1 \\mid \\mathbf{x}) = {0.1 \\times 0.1 + 0.2 \\times 0.1 + 0.8 \\times 0.6} = 0.51$$\n",
    "\n",
    "$$\\hat{y} = \\arg \\max_i \\big[p(i_0 \\mid \\mathbf{x}), p(i_1 \\mid \\mathbf{x}) \\big] = 1$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### References\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- [1] S. Raschka. [Python Machine Learning](https://github.com/rasbt/python-machine-learning-book). Packt Publishing Ltd., 2015."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 1 -  Classifying Iris Flowers Using Different Classification Models"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn import datasets\n",
    "\n",
    "iris = datasets.load_iris()\n",
    "X, y = iris.data[:, 1:3], iris.target"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "5-fold cross validation:\n",
      "\n",
      "Accuracy: 0.95 (+/- 0.04) [Logistic Regression]\n",
      "Accuracy: 0.94 (+/- 0.04) [Random Forest]\n",
      "Accuracy: 0.91 (+/- 0.04) [Naive Bayes]\n"
     ]
    }
   ],
   "source": [
    "from sklearn import model_selection\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.naive_bayes import GaussianNB \n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "import numpy as np\n",
    "\n",
    "clf1 = LogisticRegression(random_state=1)\n",
    "clf2 = RandomForestClassifier(random_state=1)\n",
    "clf3 = GaussianNB()\n",
    "\n",
    "print('5-fold cross validation:\\n')\n",
    "\n",
    "labels = ['Logistic Regression', 'Random Forest', 'Naive Bayes']\n",
    "\n",
    "for clf, label in zip([clf1, clf2, clf3], labels):\n",
    "\n",
    "    scores = model_selection.cross_val_score(clf, X, y, \n",
    "                                              cv=5, \n",
    "                                              scoring='accuracy')\n",
    "    print(\"Accuracy: %0.2f (+/- %0.2f) [%s]\"\n",
    "          % (scores.mean(), scores.std(), label))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy: 0.95 (+/- 0.04) [Logistic Regression]\n",
      "Accuracy: 0.94 (+/- 0.04) [Random Forest]\n",
      "Accuracy: 0.91 (+/- 0.04) [Naive Bayes]\n",
      "Accuracy: 0.95 (+/- 0.04) [Ensemble]\n"
     ]
    }
   ],
   "source": [
    "from mlxtend.classifier import EnsembleVoteClassifier\n",
    "\n",
    "eclf = EnsembleVoteClassifier(clfs=[clf1, clf2, clf3], weights=[1,1,1])\n",
    "\n",
    "labels = ['Logistic Regression', 'Random Forest', 'Naive Bayes', 'Ensemble']\n",
    "for clf, label in zip([clf1, clf2, clf3, eclf], labels):\n",
    "\n",
    "    scores = model_selection.cross_val_score(clf, X, y, \n",
    "                                              cv=5, \n",
    "                                              scoring='accuracy')\n",
    "    print(\"Accuracy: %0.2f (+/- %0.2f) [%s]\" \n",
    "          % (scores.mean(), scores.std(), label))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Plotting Decision Regions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/kani/Documents/KatrinaLand/mlxtend/mlxtend/plotting/decision_regions.py:249: MatplotlibDeprecationWarning: Passing unsupported keyword arguments to axis() will raise a TypeError in 3.3.\n",
      "  ax.axis(xmin=xx.min(), xmax=xx.max(), y_min=yy.min(), y_max=yy.max())\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/mlxtend/plotting/decision_regions.py:249: MatplotlibDeprecationWarning: Passing unsupported keyword arguments to axis() will raise a TypeError in 3.3.\n",
      "  ax.axis(xmin=xx.min(), xmax=xx.max(), y_min=yy.min(), y_max=yy.max())\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/mlxtend/plotting/decision_regions.py:249: MatplotlibDeprecationWarning: Passing unsupported keyword arguments to axis() will raise a TypeError in 3.3.\n",
      "  ax.axis(xmin=xx.min(), xmax=xx.max(), y_min=yy.min(), y_max=yy.max())\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/mlxtend/plotting/decision_regions.py:249: MatplotlibDeprecationWarning: Passing unsupported keyword arguments to axis() will raise a TypeError in 3.3.\n",
      "  ax.axis(xmin=xx.min(), xmax=xx.max(), y_min=yy.min(), y_max=yy.max())\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkkAAAHiCAYAAAD8s1iEAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8GearUAAAgAElEQVR4nOzdd3xb1f3/8deRZMt7xNsZtrMTVhghAQqEkZBAgDAa9mopoy2UH7QU2tL1bb9f2lIKZZVQ9gphBQijSYFASUoSCCGQkB3HTjzlLW9J5/eH5GBbV4lkS5ZkfZ6Phx8kV1fnHhn05pxzzz1Haa0RQgghhBB9mcJdASGEEEKISCSNJCGEEEIIA9JIEkIIIYQwII0kIYQQQggD0kgSQgghhDAgjSQhhBBCCAPSSIpySql/KKXuGsD7xiil7EopcyjqFamUUu8qpa4Kdz2EEMGllPqtUuq5cNdDDC/SSBpCSqlSpdTpwSxTa32D1vp/Ar221rpMa52itXYGcj2l1NVKKaengdWslPpSKTV/IHUPB631PK310+GuhxCxwJM77Z68qFJKPaWUSgl3vQZDKTVLKeXyfKaen7eG8PrFSimtlLIM1TVjmTSSxED8V2udAmQADwOLlVIZwb5IrI1yCTFMne3Ji2nAkcCdYa5PMFR4Opk9P2cHWoDkW3SQRlIEUEpZlVL3KaUqPD/3KaWsvV6/XSlV6XntWk8vYrzntaeUUn/w/DlbKbVMKdWolKpXSv1HKWVSSj0LjAHe8vR6bu/fG1FKjVBKPem5RoNSaunB6q21dgHPAsnAhF6f5R6lVJlSqtpzOzAxgM/yiFLqHaVUK3CKUqpQKfWqUqpWKbVbKXVzr7KOVUp95hnRqlZK3es5nqCUek4pVef5XaxTSuV5XluplLrW82eTUupXSqk9SqkapdQzSql0z2s9v5+rPJ/FppT65cD/LQsR27TWVcC/cDeWAFBK3aGU2qmUalFKbVZKndfrtauVUp948qTB8/2f1+v1EqXUR573rgCye19PKXWOUmqTJwNWKqWm9HqtVCn1M6XURqVUq1LqcaVUnnLfjm9RSv1bKZUZ6GdUSk3xXKvRc+1zer0WlHwDPvb8s9GT58cFWk/hP2kkRYZfAjNxh8cRwLHArwCUUnOBW4HTgfHArAOUcxuwF8gB8oBfAFprfQVQhqdHp7X+s8F7nwWSgEOAXOBvB6u0cveErgG6gT2ew3cDEz2fZTwwEvh1AJ/lUuCPQCqwGngL+NJTzmnALUqpMzzn3g/cr7VOA8YBSzzHrwLSgdFAFnAD0G5wras9P6cAY4EU4MF+53wHmOS59q97B60Qwn9KqVHAPGBHr8M7gRNxf19/BzynlCro9foMYCvuBtCfgceVUsrz2gvA557X/gf3977nWhOBF4FbcOfhO7g7ifG9yr4AmI07r84G3sWdmTm4/994MwFQSsXhzqvluDP0JuB5pdSkXqcFI99O8vwzw5Pn/w2kniIw0kiKDJcBv9da12ita3GHxRWe1xYCT2qtN2mt24DfHqCcbqAAKNJad2ut/6P92JzPE0rzgBu01g2e9350gLfMVEo1Ah3APcDlWusaT3hdB/w/rXW91roF+F/g4gA+yxta61WeUarDgByt9e+11l1a613AY73K6wbGK6WytdZ2rfWnvY5nAeO11k6t9eda62aDa10G3Ku13qW1tuO+DXCx6nuv/3da63at9Ze4w+yIA/xehBDeliqlWoByoAb4Tc8LWuuXtdYVWmuX1volYDvuTmKPPVrrxzxzJ5/GnW95SqkxwHTgLq11p9b6Y9wNjh4XAW9rrVdorbtx51QicHyvcx7QWldrrfcB/wHWaK2/0Fp3AK/jvjXoS6FntKjnZyHujm4KcLcnrz4AlgGX9HpfMPJNDCFpJEWGQr4dicHz58Jer5X3eq33n/v7C+5e2nKl1C6l1B1+Xn80UK+1bvDz/E+11hlAJvAm7p4guHtgScDnPeEBvOc5Dv59lt7HiugXRrh7enme17+Puxe4xXNLrWcC+bO4h/UXK/dtvT97enn9Gf3eLb3KB6jq9ec23CEohPDfAq11Ku6R48n0ui2mlLpSKbWh1/f7UPreNtv//fN0rMD9HSwEGrTWrb3O7f1d7vPd9jRKynGP2PSo7vXndoO/H+i7XqG1zuj1s8RzzXLPtXrXqfc1g5FvYghJIykyVOD+wvQY4zkGUAmM6vXaaF+FaK1btNa3aa3HAucAtyqlTut5+QDXLwdGqAAnX3tGX24ErlBKHQnYcIfLIb3CI90zadPfz9K7nuXA7n5hlKq1PtNz/e1a60twD23/CXhFKZXsGQn7ndZ6Ku6e43zgSoNrGf3eHfQNSyFEEHhGp5/CPaqDUqoI98jJj4EsT8fra0D5KqOXSiBTKZXc69iYXn/u8932jHKPBvYN4iMcTAUwWinV+/+rY/pdc9D5xoGzXASZNJKGXpxyTyzu+bHgvnf+K6VUjlIqG/ccnp71PpYA13gmBCYBPtdEUkrNV0qN9wRCE+AEeno11bjn3XjRWlfivh//sFIqUykVp5Q6yehcg/fWA/8Efu3pQT0G/E0pleup08he99j9/iwea4EWpdTPlVKJSimzUupQpdR0T9mXK6VyPNdt9LzHpZQ6RSl1mGfOVDPuYWuXQfkvAv9PuSeApuC+NfiS1trhz2cXQgTsPmC2UuoI3A98aKAWQCl1De6RpIPSWu8BPgN+p5SKV0p9B/e8oh5LgLOUUqd5RpFvAzpxzwMKlTW4R5tv92ToLE+dFvs4f0D5hvv35cJHnovgkkbS0HsH92hLz89vgT/g/sJvBL4C1nuOobV+F/g78CHuW2k996U7DcqeAPwbsAP/BR7WWn/oee3/cDfEGpVSPzV47xW4GxNbcM8buCWAz3QfcKZS6nDg5z31VEo1e+ozaQCfBc88hPm4J4Hvxj1S9U/ckzwB5gKblFJ23JMcL9ZatwP5wCu4G0jfAB/hvgXX3xOe4x97yu/APdlSCBECnjmXz+DuVG0G/oo7q6pxz9FZFUBxl+Ke2F2Pe57TM72usxW4HHgAd26cjfvBla4gfAxDnrLPxj2/04Z7eZQrtdZbfJw/oHzz3Hb8I7DKk+czQ/WZBCg/5vWKCOJ5uuprwBrtIx7D6bMIIYQYfmQkKQoopc5T7vWHMnHfm34rWhsVw+mzCCGEGN6kkRQdrsd9C2wn7nlGN4a3OoMynD6LEEKIYUxutwkhhBBCGJCRJCGEEEIIA9JIEkIIIYQwYDn4KYFbvOlpuYcnRAy5+JCr/FkAMCpIfgkRWyaMmMzRBTMMM0xGkoQQQgghDEgjSQghhBDCgDSShBBCCCEMSCNJCCGEEMJASCZuCyECp7QimTSsJivKr43Qh5ZG0+nqpJVmtJK5zUKIb0V6fsHAMkwaSUJEiGTSSEtKA5MmIjNGg9VlhTaw0xTu2gghIkjE5xcMKMPkdpsQEcJqskZ2wCjApN31FEKIXiI+v2BAGSaNJCEihEJFdsAA7ipGeiWFEEMtKvILAs4waSQJIfZbs3Itl596NZeefCXPP/xiuKsjhBABCXaGSSNJCAGA0+nkvl8/wJ+f+l+eXvE477/5IaXb94S7WkII4ZdQZJhM3BYiCt1w4W00NrZ5Hc/ISOIfr/x1QGV+s2ErI4sKKRxTCMCpZ8/ik+WrKJ5QNKi6CiFEb6HILwhNhkkjSYgo1NjYxsQb7vM6vu0ftwy4TFu1jdzC3P1/zynI4ZsNWwZcnhBCGAlFfkFoMkxutwkhhBBCGJBGkhACgOy8bGoqavb/vbayluy8rDDWSAgh/BeKDJNGkhACgMlHTGJv6T4qyyvp7urmg7dWcsLs48NdLSGE8EsoMkzmJAkhALBYzNzy+5v46ZV34HK6OHPhXEomFoe7WkII4ZdQZJg0koSIQhkZSYaTHDMykgZV7sxTZjDzlBmDKkMIIQ4kVPkFwc8waSQJEYUG85isEEKEUzTl10HnJCmlJimlNvT6aVZKDe45PSGEGAKSX0KIwTjoSJLWeiswDUApZQb2Aa+HuF5CCDFokl9CiMEI9Om204CdWmvZq0AIEW0kv4QQAQl0TtLFgOGOcUqp64DrAK79zdWc9t1Zg6uZOKgNn2zkvSXLqa2wkVOYzdyFc5j2ncPDXS0hIpXkVwSR/BLRwO9GklIqHjgHuNPoda31ImARwOJNT+ug1E74tOGTjbz05BKKFxRQXDyFplI7Lz25BECCRoh+JL8ii+SXiBaB3G6bB6zXWleHqjLCf+8tWU7xggIyx6VhMpvIHJdG8YIC3luyPNxVE1Hs7p/9hXOPvpCr51wb7qoEm+RXBJH8EqEQivwKpJF0CT6GqsXQq62wkV6c0udYenEKtRW2MNVIDAfzLjyDvzz9f+GuRihIfkUQyS8RCqHIL78aSUqpZGA28FpQry4GLKcwm6ZSe59jTaV2cgqzw1QjEQ6N9U385ge/oKmhKSjlHTHjcFLTU4NSVqSQ/Io8kl8CoiO//JqTpLVuBWSnywgyd+Ec9z38Be4eWFOpndKllVx0zULD82NlkmSsfM4eK15+B0f5NpYveYfvXn9JuKsTkSS/Ik+g+QWx8d2Ohc/YWzTkl6y4HaV6vjjvLVnOtopycgqzueiahYZfqFiZJBkrn7NHY30T695ZwcMXFPDDZSuYs/BM0jPTw10tIQ4qkPyC2Phux8Jn7C1a8ksaSVFs2ncO9+vL03uSJOD+5wL38QOFUrT1aAbyOaPZipff4ezxigl5CZw9vi2ie2NC9OdvfkHg323Jr8gXLfkV6GKSIgoFOkmyp0eTOTuR6b+ZQubsRF56cgkbPtk4FNUdsFiaDNrTC7vsaHegXnZ0GuveWRG0e/tCRJJAvtuSX5EvmvJLGkkxINBJktH6eG4sTQbt6YVlpbgHg7NSLJw9XrF8yTuDKvd3N/2RH55/M2W7yrlw5sW8/dK7waiuEIMSyHdb8ivyRVN+ye22GBDoJMnaChvFxVP6HEsvTmFbRflQVHfABjIZNFp9uXo9H1Z08OLGij7HR9jWD2rI+jcP/HKwVRMi6AL5bkt+Rb5oyi9pJMWAQCdJ9vRoeu6NQ3T0aAL9nNHsD0//JdxVEGLIBPLdlvyKfNGUX9JIihGBTJKMpR6NECI6+Jthkl8imKSRJLxEa48m1h6hFUJ4k/wSwSSNJGEokJGnSBFrj9AKIYxJfolgkUZSjIvG9UR8idYJm0KIgZH8EqEmjaQYNtyGd6N1wqYQInCSX2IoyDpJMSxa1xPxZe7COZQuraRhZzMup4uGnc2ULq1k7sI54a5a1KipqOEnF9/Glad/j6tmf59XnpA9YUVkkvwS/YUiv2QkKYYNt+HdaJ2wGUnMFjM/+tUNTDx0Am32Nn5w9o0cc+LRFE8oCnfVhOhD8kv0F4r8kkZSDBuOw7vROGFzoD5duZZXX3iVyvIqCkbnc8GlFzBz1rGDKjMrN4us3CwAklKSKBo3htoqmzSSRMSR/Ipu0ZJf0kiKEUYTHA+0nkikTIiMlHpEmk9XruWxRxdRfG4hY0oOpXF3C489ughg0EHTo7K8iu2bdzB12uSglCfEYPTPgomHTGDd0nURnV9G9ZYMi678kkZSDPA1wfGiaxZy0TULvYZ3gYiYEDncJmYG06svvErxuYWMGJ8O4P7nue7jwQiZttZ2fn3j77jp1z8kOTV50OUJMRhGWbBu6TqmHzudbSu2R2R++aq3ZFh05ZdfjSSlVAbwT+BQQAPf01r/d1BXFkPmQOtv3PH3n3p9We+++Z6A1uvw1VMabA9K1g3xrbK8ijElh/Y5llGSytbyPYMu29Ht4Nc3/JbTF5zGSXNPHHR54Sb5Ff18ZcG2Fdu54+8/7XNuoPkFkmFDLZryy9+RpPuB97TWFyql4oGkQV9ZDJlAJzgGcr6vntKuTbtZt3bdoHpQw21iZjAVjM6ncXfL/p4YQOPuFgpG5w+qXK01f/r5PRSNL+Kiay8cbDUjheRXlAskCwLNDcmwoRdN+XXQJQCUUunAScDjnkp0aa0bg3J1MSR6Jjj2dqAJjoGc7+sx3BVL3x/047mB1juWXHDpBZS+UUH9jiZcThf1O5oofaOCCy69YFDlfvXZ1yx/7d+s/+8XfH/e9Xx/3vV8+uGaINV66El+DQ+BZEGguSEZNvSiKb/8GUkqAWqBJ5VSRwCfAz/RWrf2PkkpdR1wHcC1v7ma0747a1AVE8ET6IaPcxfO4dlHniNrVirWHDOdtU7qVrZwxY2Xe51bW2EjsTGXT//6JW21HSTlJFA0q5CO1g7Si1P6nBtoD0o2qvSt5779qy+8ytbyPRSMzucH11836Pv5h08/jI9K/x2MKkYKya9hIJAsCCS/QDIsHKIpv/xpJFmAo4CbtNZrlFL3A3cAd/U+SWu9CFgEsHjT0zqotRSDMpD1N5wdLqo+rKPT3o01JQ5Tp/F/KlarlW1vl1J8YT4pRYnY97Sz7ZVSzGbLoB/PlXVDDmzmrGOD9iTIMCb5NQwEmgX+5hdIhoVLtOSXP42kvcBerXXPmNUruENGDJFgPEIayPob7y1ZTmJRPLVf2+lud+LscJJz6AjDyYYmiyL/pEySCq0okyKp0Er+KZk0ruikdGnloHtQsbRuiAgJya8wC9Yj8P5mQSD5BZJh4sAO2kjSWlcppcqVUpO01luB04DNoa+agPA8QrpzYymkOhl7VQFp45Jo3tnG7sVV2Fqavc5tb+1g7OEl2Jta6OxuxxIXx8jDC2lcsdtweQEJCzGUJL/CK9LzCyTDxIH5+3TbTcDznidDdgHXhK5KordwPELq0F1MuHgk6ZPc60ukT0qm5OJ8ti/a53VuTmE23TYH+ePy9h9r2NlMTmG29KACpNHuB9RVuGtyANpTz+gi+RUmkZ5fIBkWLFGRXxBwhvnVSNJabwCOGWidxMCF4xFS7dKkjEkAF+7nH12QMiYB7fL+D+tAkyRfe/QNVix9n47WDhKSE5i94DTOv/7ckNU72nW6OrG6rGDSkRk0GnApOl2dkVk/HyS/wifS8wskw4Il4vMLBpRhsuJ2hAvH/kSJKYm0lnWROjYR7XChlInWsi4SUxINzzeaJLnqnf+y8esvKbmigPTxKTTtsPOvxf8CkJDxoZVmaAOryYqKwJTRaDpdne56CuGHaMgvkAwLhkjPLxhYhimtgz90Lk+HBE/ve/pGkwf9nRDpq0dkNKly16bd/Ovtf1Fy8bfhsHtxJWecdQZjDynpc35zXQtFF+f2CcGGnc18es+XTLp+NJmTU789vqWFPc/Wcv0vr5W9jIaZiw+5KjJTcQAkv4InWPkFxhnWP48Cza+5C+fw3pLlZM5O9DvDdj1RxdipJZJfw8iEEZM5umCGYYbJSFKE8/UIKfi/P9Frj77hDo1+PaLKPVXsrS433NPtDM5gxbPv09FaTkJyAmcscAdM/2vuenQXeY3pfa6XXpyC0+EifXy/NUbGp9DeUiZ7GQkRI4KRX2CcYe+9+C/Mr1uY9qNJA86vl55cQvO+Vop/MK3P9XxlWHJhAp2dnWTOTpT8ihHSSIoCRpMHA9mfaMXS9ym5omB/jyhzcipcDJ89+jkzf3qEzz3d+g8pG12z+NwCdr5bRsHROfvPayq1Y7aYaNph79MLa9phR5mV7GUkRAwZbH6BcYYVLXSx6+mKQeUXC2DDw1sNbwkaZVjDlhaS8hIkv2KINJKiVCATIjtaOwxHdZwOl88VZY2GtmsrbORml1BVVo2juxtLXBwjxqWz/bm9NOxs7jOcfswJR7Nx8ZdwMX2GvOMt8YNexVYIEd0CndBtlGGpxQk4Op2GZfibXynZqVjMFsP1kIwybM8bNUw6v9jveovoJ42kKBXIhMiE5ATDUR2zxWRYhrPTZXh7Lt5lZd/GCkYclka8NRFnp5Par+oYkZNJw4p2r7VEXnv0Da8h722btg/5RE4hRGQJdEK3UYa1lHZgsZr7nBdofu3bWEF6VhoXXnO+4XpI/TMs3ZpOen6q1zUlv4YvaSRFqUD2BJq94DT3Uxn9RnWOOeFoSpeWe5XR1tHK2Gu8b89tfnAPjhXdaCAhN46Omm4qV9SRac3ijr//1Ou6519/rteQ94ZPNvLU357BZXX0eZLk6v93ZUh+T0KIyBPonmZGGbZnSTXxFqvXKPZA8svXekj9M0zyK/ZIIylKBbInUM+XvP+oTu+n23qXcf8vHjS8PedyunB1a6pW1tNtdxCXYsHVrWmyB/ZIuDnBRO6srD5rkgghYkege5oZZdjcBd8+rSb5JUJFGklRLJDVYI1GdXyV4fP2XLyJkovyDR/r99d7S5Yz8eKib4fZJ0JWfrNMfBQixgS6mvWBMqw3yS8RTNJIGoYGu6Gkr9tz8XHxxKdacHQ4MVvNODudxKda0E7N3Tff49f1wrECrxAiegx1flnMFr+vKfkVe6SRNMwEY0NJX7fntm3ajrnNhDPOtX8jSFeDwonT73VDwrECrxAiOgx1fpnbrKRnpfl9Tcmv2GMKdwVEcPXeUNJkNrnXMlpQwHtLlg+67LkL51C1vB5rdyKFxYVYuxPZ+eo+Ss4Y6ff15i6cQ+nSShp2NuNyumjY2Uzp0krmLpwz6PoJIaLbUOdX1fJ6XA7t9zUlv2KPjCQNM8EYDva1QvcZnMFF1yzsM1HS1Glh7JxRfl8v0AmbQojYMdT5ddE1C3n2vhf8XrtN8iv2SCNpmAnGcLCvFbpXPPs+j1z/9z6BcPfN9wR8vUAnbAohYsNQ5xe4GzyBXFPyK7ZII2mYmbtwDo/96XGcVieO1m4syXGYO81MPWwqN8672WuDWyMdrR04O1xsemA3HbYuErLjyTtxBB2tHV4THCceMoFVi1eRNSu1zyOxV9x4+RB/ciFEtPOVXz/4+fd9btLdXyD5NXfhHOYunMOzjzwnGSYMSSNpmNm1aTfdupuRp2eRmBtPe00X5W/U8vma9Uz8/qg+w8+AYciYzRb2/quWogvzSBmTgL2sgz2vVKNQXhMcVy1eRVtdB90fdvVZXE0IIQJllF/7ltWx7Kl3KK8p87qFBt4ZFkh+vfTkEqYfOx1nh4uqD+skw4QXv/5LUEqVAi2AE3BorY8JZaWGk8E+zgr43YMC91Dz2CsKSClJIN4MSQUJaKem6qMGw+Fno3LSs9JIPTmOpAIrKEgqsJJ3ciblS2xeG0RmzUql+8MuvvOro/e/v2GnrBsiIofk18BFQn5Z0+L55pEdTLmxyK8MCyS/WAArHn6faT+c1Od2m2SY6BFIc/kUrbUtZDUZhoLxOKuvSYhgPArU0dpBSnEiZqUxKTArSMyLx9nWbyPI8Sl0tPqeDJk/OYcWWzPapVEmRf7kHPa4qr0mOFpzzHTau/uWLeuGiMgj+RWgSMmvlOJEtFMbrqLtK8P8za/04hT35rmy6bbwQZYACKFgPM66Yun7lFzsnoRosigyJ6dScnEBK5a+b3i+NclK8w47Fs+/WYsJ2qu7MCf12whyh52E5ATDMnIKs9HNkGCxMCnXSoLFgm5W7pVsS+19zu2sdWJNietbtqwbIkTUi5T8at5hR5kVTTv6Zo+vDOuoa2Pv6xXUfthA95omaj9sYOtju4mLj/PKr6ZSu2GuSYaJHv6OJGlguVJKA49qrRf1P0EpdR1wHcC1v7ma0747K2iVjFbBeJy1o7WD9PEpaA2OLgeWeMsBe1DFRaPYuXg3jnNySMyLp726i71v1eLoclG9tp6EvHg6qrvYu8zG3AVnGJYxd+Ec/vnXx8mfncbOrnjaqrqoWlHJ7AWnsW7pOlxnO3HEdWLpTqBuZQumTovXJpO+NqoUgdmztZwmW2PA72utb6F5dw1mc2D9oM6OTtJcmgRrYHMyLn70qoDOH2KSXwMQKflV/mYtmakZ7HqhklHzuw+aYScfM5XVOzaTPTuNpHwruqqTiq31jM4p4MsHt5I+MRmn2YHFaaFxWytjR47kywe3kjk5hYSMeDoau2jYYmf6tEP590Nv9ik7PjOFky491e/PL6Kfv0n4Ha31PqVULrBCKbVFa/1x7xM8wbMIYPGmp3WQ6xmVgvE4a88+RGnjkkC7cDmcNO9s8zkK1N5ox9nhYt/yOhztTiyJZpydLnBA5b/r6W51EJdsIU7FMfaQEsMy2uztdDZ3YPvIQWerE2uymc5mBwVFBVx0yEKev/8FGqvqyMjP4oqfuJ8AiaZ1Q1wuF9s27MDldAX0Pq01e9Zuw6JUYBfU4GhqISstKbD3AeNGpHBScW7A74vPTeOoEw9BBVrX4UnyawCM8qtuSyO63cnyfo0Hnxyw9YkyrCPi0C4XymSis74bHLD8oTdxOl1MPW0aRYcUAz7yq8OFJRHiVJxfGba7uoH6ilbsL7fj7HJhjjfR1eri1iuOpTA7nVsXvca+mkZG5mby/K2XMm/mVN79dDP3vbGS3V/VMz5/BLfceg7zZk71Kvv2Zz7w+uwul4uJJx7G2Gnj/P/liqjhVyNJa73P888apdTrwLHAxwd+l5i7cI77Hv4CBjzKMnvBaby3+F8UX5jLiPFJ1G9tofSVGp+jQIl56cy4PL9PsO1YVUrZm1UkpFtxtDlJSE8gZ2qmz4mJrz/+OqNPSKelrIOuNhdxiWZGn5DMa4+/xl2LfkmeRfHKDUX8aFkr4w4dS2pGCmMPKeGRnz3I3IvmkpyayI6NO/3+jJWby2irC2wnboCOplbyUgNveDgcTk4el0d2inFD80COXDCDjAFcU4RPoPkVyH+7w01BcT7JacmAcX7teq2CK04+ij9dPsuv8kocHfz13yvJPT7dnV872ih7uYZfXHQ6d10+i8aWNn7+wVf7G0lG+dWws5kND29l9Kl51G6uP2iG2dqbGH9GFi1lHbTWdZOcFYc2K/7wwnuse/hnFFgUS68bxY3L2jh2ShEA82ZOZfrkMVx/93MsuuMKstKTDT/Pn6/0HkXq6Ozmh2+ukUbSMHXQRpJSKhkwaa1bPH+eA/w+5DUbBoKxOuvcy+bw5X82sGtROducGotZMbJ4NEeffCR7tpR5nV+xq5JMNZbqDXU0bm0CBVXbbbTVd5AwOp7Mw5LpanGy890ydKvivX69ova2dqp21WCzmUmblOxDjwEAACAASURBVEzy1GS6mhzs/k8DjmYnz/70ETK67Ly13kFGVwfP3PYwR02byPoN22jZVUrNW//hvFOOCuj3dN6kQg4Ze/TBTxQiQAPJr2mVsTm/W2t49PVVnHjRLAAyszM4ZfYprHptNZtqd5GZk8H0mdOZVpzvd5l3XT2X1V/uYPUTu9nh0MRZFMePLeGuq+d+e1Kv0U5ft/haG9uo3FDLqPlZJBXm0lbRRdnSahzVmj1bykjPTicjO91TRh2tZiujz8lh9OgEWss72PlcJc46B8+8vZr5401MyrUyf3wHTy9bxa2XubcUeebt1TRUlfc55q/6mkavPLbEWRg5rjCgckTk8WckKQ943TNsbwFe0Fq/F9JahZjWmqqyarQrsFF1p9PF5g834Op2HvzkXo6YPJH2/ELyU5Np+2I3q7/Y7fd72+2txNU18ODcHFITTLR0uLhnVQPZG7aTanDLrSA5gdaNdWSMTmHyBWNQSvHa72oZf8VIso5IId6s6HJq2nZ1svfZGp4y6BHmvf8Jo67MJXNS6v5jDVtbKH+6mjh7HYsvzSM7xYLN7mDhkjp+NOtQbvjkv7z9vUJuXFbKcT9c4LMnJsQQCzi/5s+ccqCXh7WEeAsN27+dL1SSnMDc878dPVEo5h493u/ybI12Olub2HlTUa/MaKKuqdUwI3xNUTDHmxg5L4vkMe6lAdSYBCwpJpwVHZRsL+f111dx7p2XAJCYlsiYc3L2LxdgnWyleXo7TR83seyjdSxZ6D5+5VHJLFyyjqvmn4DWmmUfreOR87O5cZn7mL8ZlmCN44ZjJ9C8ve88q3c/207ij85lRG6m378vEXkO2kjSWu8Cjgik0NqKwHpipRt20lhZH9B7ANqa20gD4uICm2Ta1dnN4XnppCZZA77mVbOnkZORcvATDQxkfsi9zy9nylGJXD09ff+x+nZoqm/g+/O8ezup8WZuX/wWBVMz0C6NrbSFjpouEnPj0d0aTKC7NaZkE/Fm9xNvtka7e5j5Tvcwc7zZTFyqBUenE0u8CUeXy/33bhfzx7snAl/wZDmLFhYyf7yJnz/4MvPGgrmrmXlj4/r0xPqXLcRQGkh+xbLZR08Iank9IzfZKe6Mzk6xMH+8yedoja8pClarFUuyCbpdYFLQrTEnmkhNSeDUw0u44x9vcmrj2aRmpGAxW4hPteDocGK2mnF2OrEkmnA4tGF+Pb1sFcCgMmzu9Elex2rtHegA5z2KyBOSZUUtH28M6PxzR2cz/ZjpAV9HoUiwxh38xCi2cv02Kmo6eeGrmj7HC6u3GYZMz2TD+95YySdVeynJH8HEkTl0V3aRGAd2J8SZoXNfFxNG5gDew8xTi/JxtFpwxLmwd3dhjbOQ3BqH1WLmha86eXB1IxmWLqY/sJe0JCvtbfX84bIMnN1dnDnOwk0rvu2JDWYIWwgR3Q6WXxazmW3rt/W+48bUoolsfOIbWprspKancPiRU9hg28S+d2pJzLTgdIHZBG3lHRw1ciTPvL2ajsZ6nrzrnxw+41AS4qzUftgCiS6cThdms4m20k6sFpNXfo1ITSRr7zd0d9h5YDZBzbDkeAvLX/0PaX52qvPGFjLl+L6TxZ0OJ6tf/Q/Oboff1xWB6zi6laMvnGH4WkgaSTfOkwVtg+XNv/444PfMmzm1z5MZ1/3fc7z7768ZeV42uWPiqSnrYs+/bcycdCi2RjtvfLCWXx5v5o8fruWq+Sdwy7mzuH3xWxSdnQvxmtS2ePb8q4YnbruM6ZPHsPD2+3lkfhI3LmvjlBlHkFD7JRZnB0WZFvY0djBvbCJPL1vFlWcdP+AhbCFE9Lv6/FMpq2vxOn7C5FEApCRZefNnF+LoPeJy8qFwTd/GyC2Njfx7xzeMPTqF7FFx2PZ20/lNK6NSUnjjg7UsOjuJ/127l58deQ6nJsfx29feY/TcHFriukjtjqe8zMZf77jIK79e/sstPL1sFc7ydUHPsKtOm8a5Le1+/65+9soqr0ZSU30zI5vt3DhX/p8aStbRvifdywY1MaDK1ojL5mTVogqcWmNWilRlpiqrkWfeXs3JI7soSnVycmFXn96S+1HZBkbmZnLvdeczb+ZU7n1+eZ+Jj8988Dlt7e08trqLtARFc4cGSzeTa7cB+JwkKYQY/l7ZVM6JV8/2Ov7Uc+8zfeJIAMO5lf01NNuhwcXaZ6r6ZNhGVznnjHUwIcPF6aMdvPHRem69bA6pSVa/8uvpZatYuX4bW3a3BD3DlFJkBrDsR3yc8V2RlKSEgMoRA5Do+79BWXE7Bjxx19WMz0tn+YWJ1N2cxvILExmfn849P1nIGx+s5cwSJ0WZFs4scfLGh2upa2pl+uQxFFgU664bRYFFceyUImyNdpZ9tI4rj3L3pK48Kpn0BDN5GcmsvrmYz24dx+qbixmVnco9P1node6yj9ZR19Qazl+FEGIIJSVZyczJ8PqxWMwHf3MvRhlWkpNKkoVB5deyj9Zxz08WMio7VTJMGJJG0hCwNdq54I5/DPjL9e6nmznjzoeZeM0fOOPOh3n3080Bvb9ntGhMuom9TQ7GZJg4ubCLnz/4MieP7CLVqrjsFTtpCWr/aNIzb69mTrGms9nGnGK9/1j/SZgnj+wiO64DcE+GVErtn8zta8JmMH4nQoihMdjv6oZPNnL3zfdw24V3cPfN97Dhk8DmrIJxhh2X2066sg8qv3pnFfifYZJfsUNutw2BwUxefvfTzdy++C3GnpvHd4pzqSu1c/vitwAMV4Q1smLtN3y9vYUXv4AOh4sEi4kOJ2Dq4DNXN89+rslMgLOes+PQiom2r9GOdu4+sRuLcnHq6G7u+Ggd8Qkp2Br6TsKsaWil2wnHPrivz2RIW3M55ZUJPidsyoRuIaLDYL6r5WXVvL9+rdcmuYePGB1QOUYZ1tLpotsF85/vGnB+Afuzqv+E7gNlGCD5FSOkkRRiPUO8RhP//Hm09L43VjL23DxyPOuG5IxLg3Pdx+fNnOpXGbOPncJx2XZWbmvkkbMSufHtdk6ZlMH65gyOTG00OG5lRkYbiaqLkakmypq7mFMcR3zRFG69bI7XNW2Ndq/JkAea3Hig34kQInIMNr82bdrOhOuL9q97lDkuDRbAhkd3+F0GGGdYkjWOydkm1u/rHFR+9dTD3wzbf67kV0yQ220h1neF129vN/W81tMb8WV3VT1ZxX0fIc0qTmF3Vb3fZaxcv42nPm/miDxwahdH5MGTnzXzxdZyn8cfWd3AgsVtnPRUKwsWt/HI6gZWrt9meM0DfcZAfydCiMgx2PxqaWklvV9+pRen0NzS5ncZYJxha/a08ewX9kHn18E+ZyC/EzH8SCMphHxNFKxrau3TQzvQZMCS/BHUldr7HKsrtVOSP8LvMp6462qKc9P45dzRTC0p5JdzR1Ocm8byB241PL74jzcQZ9I8d14Sa65N4bnzkogzaf56y0Kva24rqwlocuOBfidCiMgRjPzKSU9h02Nb2bFk1/6fTY9tZXRept9lgHGGTRqdxeTR2YPKr96fxZ9MkvyKPebf/va3wS+1fG0ICo0+/3h1JRPjqjhtgvvxzaR4E3UtXXxZ5eDL7eVMjKvi7KnJ+48dd7j3Wg3ZKckseXU91tw4EtPjsO1uYdcb1dx14Rm8v2aTX2X4qsfjK3cye1QXU3Pj+O6TpZx7WDqdXQ7+97UvOKu4mzMnWMhIMBFnhoZ2zUtrK2hsbutzzZ4yjD5jIHXxdb6IEqNn/C7cVQgayS8gOPk1oTCbD9dvY+ysbMbPzMGaYKZlYyt/vvJsv/PLV13e2tjAsQWao0YlDji/en8WfzIplPn19sZSio7tu3J3e2sHrl2VzPAslyBCJCUPMosNM0xpHdj+ZX5Z/UAICo0+59z2IBU13lu0ZGVm0t1hZ8nC1F77GbX4vA/+7qebue+NleyuqqckfwS3nDtr/6Jo/pThqx625g6y0xKorGshJ8FJbYeZgqxUdlc1Em/SZCcpTApcGmxtGkuclQmF6X2uecKD5SQmJWIx9x2ULMzNNlwI01ddfJ0vosTxNwW+506kkvwCIie/fNVln62FODM4XQw4vxYuafFM6G7wuqZRJoUyv3747EpO+vE5fY7V1zTgWPE5N88/dlBli4PIPxzGnmyYYdJICoN7n18O+z7n1pO+3Y/t3o+bYOTRfj8pEYwyALbuqeasm/7EawuTOH9JG+8+dAdvfbzBsOz/2NI4Mbt50NcUw5A0kmKG5FdofP/xFUy/8vQ+xxpqG0lev10aSaEmjaTIEozeyMHK8PepkfNuf4hDzHu4elocT23oZpOzCKdTU1Fjo6vbSbmtmdHZacTHmfePPA2m3mKYkkZSzJD8Co0VX+ziq7Iar+MLj5vMqNyMMNQohkgjKfbc+/xylq34iPmzT/bZS+rphS1dmECcZ4PtBUs6ePehO5gwOpfzbn+Isl07GDN2PK//+UdD/AlEVJFGkggiyS8xpA7QSJKn24ahnk1r7zjevH+Z/p7jvVeJveOhV7j4EAtxZijKcE9wvPgQC7c/8DJb91Tz1Zad/PHUBL7aspPt5d49HCGECDZf+dXzWk+GSX6JoSCNpGHIaNPanuO91wf5Yms5j6/v4sznW5nxTztnPt/K4+u7+GJrOXc89ArnTTJTkqE4b5KZ2x94OZwfSQgRI3zlV89rPRkm+SWGgt+NJKWUWSn1hVJqWSgrJAanpxfWf9PHnvWMeq8P8tnTv2JKUQ5rbhnHl7dPZM0t45hSlMPiP17PV1t2cv5kC0UZJs6fbJHemIhqkl/RwVd+Ga3NtPyBWyW/RMgFMpL0E+CbUFVkOPO1GWIgmyT6W0ZPL6w400yCxURxpnn/ZrbzxoK5q5l5Y9UBN3y88U/PunthmSYSLIqSTJP0xkS0k/waoEjIr5686p1hvjaglfwSweTX3m1KqVHAWcAfgVtDWqNhyNcGkYFsHOlvGT0bQS7bbMJkApcLquxOUqxt/OHyTJzdXZw5zsJNK3xv+FhW1chzNZr3dnT3WWckLr4suL8YIYaA5NfghDu/attc+zetfWA2+zPsn5+WsntvIi981dnnWpJfIpj83eD2PuB2IDWEdRmWfG0QGcgmrz1D0L883swfP1x7wDJmHzuF2SPbuOSIJK55oYynLxvDDa/WcEiOwuLsoCjTwp7GDuaNTcQ8eophuB1oDRMhopDk1wBFQn49v6GN/9isHD+ivU+GXTszE/PoY7wyTPJLBNNBb7cppeYDNVrrzw9y3nVKqc+UUp8tekM2/OvhazPEQDdU9DURu38ZK9dv44WvOjnm/jKqWpwcdV8Zn5Z18ORnrZz9QivfecLO2S+08tja5v0bPva3v4yHavb/vPBVp8/zhYhUkl+DEwn59cJXnXyxtZzH1rb4lWGSXyKY/BlJOgE4Ryl1JpAApCmlntNaX977JK31ImARIOuMePT0lJYsdHdgrzwqmYVL1jH/xGmGx416Yz29sD99p2cio4Off7jWZxkv/+UWbI12zrrpTzxzfgrnL2nj+f+7mTvvf95rOf4nf32NYb0jYWE1EdkqbU38d0slX5XVUWrr4Mnjbwp3lXyR/BqgSMmvF//3h2SmJhluY2KUYZJfIpgOOpKktb5Taz1Ka10MXAx80D9ghDFfE6N9TTh8etmqgCZi+yrjjode4dJDLRyeH8elh1q44e5nfJ4rxME0trSxc28t97y6hh889D43PL6GO9+zsSbuGLLOuYuZN9wT7ir6JPk1cAPJL+g7GTsY+XX7Ay/7rItkmAg1f+ckiQFYuX4bFTXeE6NtzeWUVyZ4HS+sdg8H+zOR0WTppDw90auMjD1fU1q+l4e/lwLAjccm8PSGep5an+o1wbGweltE7FkkIkd7ZxctbZ28umorm/a14LIk0aSTSM3KZ+xxN3JUZhaWuPhwV1MMgYHk162XzekzGTsY+XXiEztpbu2kqdm7LpJhItRkW5IIYmu0s+DWv5GsW2lTySz9263unlIAG0Ged/tDHGbZw+9P/XbY+9cftPKVo0iW5hd9aK1p6+ji1U+2sK/OTrld0dgdR2JmLkXTz2BEbgHWxCS/yvrBSWNlWxLhlWFzTjiS5LqvJL9EZDvAtiQykhRBnnl7NTlxHTS1dpOd3LF/IqNRb85XD+qLreWs7erm8S8a+xyPiy8Pad1F5HO5XLy+agtVDa1squqkqb2buIwCRh51FmmH5HBkbmG4qyiiXP8Me/WDz7GYtOSXiFrSSIoQtkY7r72/BnNnJ4vOTua6t9p4/f01LP3brQfcBbu/0jfuDmEtRTTZsqeatdsqWbO7iZZ2B67kHAoOP4m0QwuYNq8EpYbP4I8IP6MMS7UmBJRhkl8i0kgjKUL09MBOLI5jUraZcyfF8Z99HX4t1DZYtkY719/9HIvuvCKgBpmIHOXVDXy+bR/vf2PD3unElZSNNT2XvKkLOPzkKZhMsk2jCK1wZZjklwglaSRFiH+t+YbtFe38dEYCm2u7mTFK8eC6dtrXfBPyRlIgK+eK8KttaOGbPdWs3lbLjupWuhNGYEnOIGfq6Uy94jCZWC3CIlwZJvklQkkaSQMQip7LGTOmcMaoNk6Y+u0Exx/bmmDklJDWJZCVc8XQa25tZ8ueGtZsr2VPXRuVLU4s6XlkTT6e7JmjmVE4JtxVFFEoUjJM8ktEOmkkDUAoei6BTtAOVl36rno7NLf3hG/bympYtaWSb/Y2UdniQCVlMmLyTDKnnMaksZOYFO4KimEhUjJM8ktEOmkkBShUPZeBrBI72Lr4WlFXemNDo7y6gbXbKvlsp42Kpi7ikzMgcwx5E+Yy4dSjmBDuCophKVIyTPJLRANpJAUoknoug63LgVaxld5YcNka7TS0tPHyqh3sqm3DkpxBpzWL3PEzGHvpdyhRSp42E0MiUjJM8ktEA2kkBSCSei7BqMtAb/GJA2tp7aClrZOXV22jst5OgzORdksaydkjGXvKTzgmfQRmi3z1xNCLlAyT/BLRQpI6AMHuuRhNWvR3ImMw6iIbQQ6ey+WiubWD11Zvo6K+lbIWaCOR5OyRjD7yKgoysxmXkhruagoBBDfDfGWVPxkm+SWihTSSAhDsnovRpEV/JzJKL2roaa3p6nbw5qfbqWxoZVNlO21dLuKziyg4fD4Z0wqZnpUT7moK4VMwc8NXVvmTYZJfIlpIIykAwey5GE1a1Fr7PZFRelFDY2t5Las37WXN7ibaHRqdmk/BIceRUVLE0fmjZB6RiCrByg1fk679nYwt+SWihTSSwsRo0iIQERMqY9XemgbWbatkfWk9FY1ddMRnkpiZx+gjL2LarBKZRySEh69J15EyKVyIYJHUDwOjSYsXvLgWl9a8fmn6/mPyOGvo1DW1sm5LORv2NLC7roMWlYo1JYORR84lfUIeY7Jyw11FISKSr0nX80+cFhGTwoUIJmkkhYHRpMWTR3bxVbWT7JSs/cfkcdbgsLd1sn5bOV+WNVBa10llczdxabnkHXEqGTNzOUZWrRbCb74mXf/8wZdj/pH8uqZWfv/SWixxceGuigjAjONg4diTDV+TRlIYGE1arGlopdsJxzwkExkH65vSKtZsq+Lrvc1UNXejkkeQc+jJpE3OZfLYSUwOdwWFiGK+Jl3bmsspr0yI6cnYuypsqClnMPm42eGuigjAqMI0n68dtJGklEoAPgasnvNf0Vr/Jmi1i0EyaTF4SivrsDW28sqa3VQ2dROfmgUZo8gZP49Jpx4l23jEOMmv4Ivl/LI12qmwNfl8fWtZDSp54hDWSISaPyNJncCpWmu7UioO+EQp9a7W+tMQ1030EooNKaNNdX0zdU2tvLx6B6W2duJSs3AkZpGSX8L48y5nXLKsRyS8SH5FgOGSX794fg2JU0/z+bqOH8XEI44fwhqJUDtoI0lrrQG7569xnh8dykoJb6HYkDKSNba00Whv57X/7qCqoY06hxVnwghS8ospPv02jhuRLY/fi4OS/IoMwyW/EpOSOfTEs8JdDTGE/JqTpJQyA58D44GHtNZrQlor0UeoNqSMFC6Xi9pGO2+t2UlFfSt7mjXdllRS8osZedQ1FOUUMMGaEO5qiigl+RVewz2/xPDmVyNJa+0EpimlMoDXlVKHaq2/7n2OUuo64DqAR2+/iOvOPSHolY1Vw2ntEa01LW0dbCmv55NNe9lc3U6XS2HNG0/BIecwYmYRM1LSZJRIBI3kV3gNp/wSsSegp9u01o1KqQ+BucDX/V5bBCwCYPUDMpwdJJGyIeVAdXU72FXZwMdfl/N5aTNdWqHTCkkekUfxCT9i+ogcTCZTuKspYoDk19CL9vwSwp+n23KAbk/AJAKzgT+FvGYCCP6muqFWaWviw41lbN7XRI3dRaslnaSMHIqPvZJppxZiscj6IWLoSH6FV7TllxD9+TOSVAA87bmvbwKWaK2XhbZaokckbwTZ0NzGyo2lbNnXRFljNw2OBBLSMimecR4Zh4ygSDZ7FeEn+RVGkZxfQvjDn6fbNgJHDkFdhIFIWpPk002lrNleQ2l9N9UtXVjSchl1zBmkFmVxVMGocFdPCC+SX+EVSfklxEDIitvC0Ne7Kli3rZqvKuzUNHcRn5ZNQsFE8g6dw9TiCUwNdwWFEEKIEJNGkmDXPhvVDS288VkZlU3dxKVmYcocQ874c5hy+uFMCXcFhRBCiDCQRlKMqbQ1UV3fwtJ1pdQ1d9CskrFkjCJt1BSKzr2Ccanp4a6iEEIIERGkkTSMORxOdu6z8da63VTUt1LXbcWUmkPa6EmMOm0BI7Pzwl1FIYQQImJJI2mYcLlc7K1p5F9f7KHM1kJ5k8ZliiO1+AhGHvM9JhUWyXpEQohh76q/vUtSRm5IynZYs0NSrohc0kiKQlprbI12dlc18sHGvWyt6UCb44kvmEDhpHPIO3ECo60Jsmq1ECLmqPSRHHXFneGuhhgmpJEUBextneytbeTfX+5lZ3UzLc44HKkjSc7MZfzpFzEzNV1GiYQQQoggk0ZShHE6XdQ0tPDOZ7vZVd1MXZcZuymNpMxcxs28lpLkVBKTU8NdTSGGra5uR7irIAZIa412ucJdDTGMSCMpzFpaO3hrzXZ2VTdT026i1u7AmjWSccddQtYx6RRnZIW7ikLElOtf3BnuKogB0loz9rgzw10NMYxII2mIrf56D2u3V7G7wUGtZ9XqMTPmkzJpBIflFYa7ekLEvOMvvTXcVRBCRAhpJIXQV7sqKK9pYsXX1dhaXZhTs0kunEDe4WdwaNG4cFdPCCGEEAcgjaQg2bXPxt7aRt7dUMG+xk4sabnEjxhFRvEpjF04ialJKeGuohBCCCECII2kAai0NVFe08h7G/ZSZ+/C1h1PQtYYMkqOYeT8KYzNGBHuKgohhBBikKSRdBBOp4vNpVUs31BORUMb1e1m4jPySS85jIJZ53FobkG4qyiEEEKIEJBGUi9aa3bts7G7upkVG8upbAEVbyWl+EgKjjqVyaPHMTnclRRCCCHEkIjpRlJFbRN7bc2890UZu2ydmKyJWHInkpQ5gcmX/JBxZrOsWi2EEELEqIM2kpRSo4FngDxAA4u01veHumLB1tDcRk1DC299todym512UyLdyQUkZeYy6azLOC4xWVatFmKYGS75JYQID39GkhzAbVrr9UqpVOBzpdQKrfXmENdtwLodTmob7Cxds4Oy2haanFbs5nSSM3MY950fMT4xmYSk5HBXUwgRelGXX0KIyHHQRpLWuhKo9Py5RSn1DTASiIiQcblcOF2aDzbsZu22KqrbzTS0O0nILmLszCvJS01nXHpmuKsphAiDSM8vIURkC2hOklKqGDgSWBOKyvhr9aYy1myrZGedg4bWbuIy8hlRPJX8Uy7iiJz8cFZNHMT//fgS7PYWr+MpKanc+eCLEVu2iH6Rkl8iekl+xR6/G0lKqRTgVeAWrXWzwevXAdcBPHr7RVx37glBqeDGXZXs3NfAx1trsLW6UCk5pI0cR/6RczliVElQriGGjt3ewthrH/A6vuufN0V02SK6BZJfl9/2B04655IhrqGIBpJfscevRpJSKg53wDyvtX7N6Byt9SJgEQCrH9ADqcyufTa27bXx8ZYa9jV2oVJySMgqJHvCXIovLGZqSupAihVCxLBA8+uxj3cNKL+EEMOPP0+3KeBx4But9b3BunC3w8maTaV8uLma+lYHlW2KlNwisiadTO6ZxZRkZgfrUkKIGBWq/BJCxAZ/RpJOAK4AvlJKbfAc+4XW+h1/L+Jyufhi214+2lzJvoYOKu2auMQU0icfT+FJFzI1r5CpA6m9EEIc2KDzSwgRu/x5uu0TIKAVFb8prWJfXQvvbaigyu7EkphKUvE0CqfNYfIYWbU6FviahFhfU4m1dLvX8aY626Cv2VRnY59B2fU1lfzy6vlex2VC5PA3kPwSAowzrL6mCsuOzZgtcX2OS34NXyFZcftvG+JJHjGVyVfczCRZoDEm+ZqEWPc/51G3zPuuh3Y5Bn1N7XIYl+10yoRIIURAjDKs+aEfU/Py74hP7buJueTX8BWSRtL0+VeEolgxDJji4jn8x494HQ/GFz4jJ98wTD6/+6JBly2EEDln/YS6Zfd6ZZjk1/AV03u3CWO+bpW11NeSOiKnz7GmOhva5SCj3/pUjbVVhmW7ujtZf9+1XscdzbWDHlJurK1i44M3Glyzy6/3CyGiXyD5daDjRhlWs+w+nC11Xhkm+TV8SSNJePF1q+zzuy/yOr6vdDt1y+71Ou6z92OyUPi9B70Olz9w+aCHlLUyUXj1fV7Hd//9Mr/LEEJEt0Dy62DH+9OOTnIv+gPx2WP6HJf8Gr6kkST85uwyHgVytjb4XYZ2drP3oSuNXjGctGg0IfLGM6ahzWbvEhwO2mvKDIr2f9mbYK16K6vnChFZfOZXV6ffZTiabdS89CuDV/zPL/CRYRrKn7qV7DNv+oYT5gAAIABJREFU6Xc8sGW7gpE9kl/fkkaS8Jsye48Cubo7qVj0A69zTcpk2ItS5jhG3viU1/G9D12JtV/vDMClXV7HtNnM6B8/53W8/IHLsWTkGdXcsC4pBouTBmvVW1k9V4jIYpRf4M4NI0YZpsyWQecXGGdYl60M25t/Nsgw//MLgpM9kl/fkkaSCIn0rGz++NQyr+M3zDsaFcInHuPirV7HTGazYV2EEMIXowwLdX6Bd4ZJfoWXNJJimK8hVVtFOY0Gw9La6f2Ya83Lv0G7XF4TDjsbjNf20E4Hrm7j4e2uzo4+f3c01eByOvnh/GP7FTL0u0Y01lbJWiVCRJBg5FfPcaMJ074yrOqFO8j97u+8jvfPLwCX0+GdXzDkGbbpn7fRYavx+jySXwcnjaQY5nMtoz9eQPY5t3sdr37xF3TZ+s75cXXYyVv4e8xxfRdXq3zhFz7KvhBljvM6rp0Oql/4eZ9jTns9lvQ8Rl/dd+2Q3ff7msioqXjqFq+jyseQdyC0MsnwsxARJBj5Be5baFnzb/U6bpRhlu2bqXv3714ZZpRf7uNOin7ygtdxwwxTCmdro1eGBSO/nB1t5F/8B0YWT+hzXPLr4KSRFAbROylOez3V4epwfw6no28vTR+gp+RruNppr+9bhssFBNLjUoZrMH3xp0tkFEiIIBlO+dVzvH9+wQEyTBlnWP/8CpQlLQ9lMnllmORXeEkjKQwif1Kc8hEmivIH+01y1GAZUYgpLsHPsrXh023KbKbwusf6HHM0VWN788/eJTgd3vUAcHYb/g61y+H37zslJdXwuEkFNg/BVzm+JlsKES2GVX71nJ9b4lfJGo2j2eaVYUb5BbDvH983Lscow7T76d/+v8dA8guMs6fbXo/Z4IlgXyS/viWNJBGQf7zzeZ+/33Dm0QE0kAClKL7pWa/Du++/DFOc96RrI+Z4Kw8vW+v3JY16Yb746pkFUsaByhFChE///AJ3hvlLKROWtGxGX3N/n+OB5BcElmHByJ5fXj2f/NFjB1VGrJJGkhgcHb0rwjbV2WQYW4hYF6UZJvk1NKSRFMN8Dalqp4OKJ37s/QaDTRy1y0H1C96TJI2GjQGU08meB40Wk8TgqTf3RMb+51tMwdnU3aVdg74NF4vDz0JEgmDkFwSWYfU1lWgfGWb81K42PDcYGSb5NTSkkRTDfPU2fnz2THIXeD+pUfPSr72OKbOFgqu8l9Ivf/DygNb2+OH8Y4m39r1tF587BpPZHNCttVCRnpkQkSUY+QXByTCj/AIwmS2SX1FOGklhEOmteu1yULfsXsPjXpxO416b0xnQNS0mFbIel6/fdzAerRUi1gyr/IKgZJjk1/B10EaSUuoJYD5Qo7U+NPRVGv4ivVWfkZPv9zBuVsGooDzp8vc31wR0fiCCNRlbRCfJsOAaTvkFwckwya/hy5+RpKeAB4FnQlsVIfqK3vVYRIR5CskwEQaSYdHvoI0krfXHSqni0FdFRKNQDr2Hej2WSL9tIIJDMkwcSLRmmOTX0JA5SWJQork3FM11F0IER7TmQLTWO9oErZGklLoOuA7g8tv+wEnnXBKsosUQkx6KiDWSX8OH5JcIpqA1krTWi4BFAI99vGvot2kXQSM9FBFrJL+GD8kvEUyBbUglhBBCCBEj/FkC4EVgFpCtlNoL/EZr/XioKyaEDJuLYJAME+EiGRb9lNbBH1mW4WohYssPThobnL1iIoDklxCx5ZDCNI4fn22YYXK7TQghhBDCgDSShBBCCCEMSCNJCCGEEMKANJKEEEIIIQxII0kIIYQQwoA0koQQQgghDEgjSQghhBDCgDSShBBCCCEMSCNJCCGEEMKANJKEEEIIIQxII0kIIYQQwoA0koQQQgghDEgjSQghhBDCgDSShBBCCCEMSCNJCCGEEMKANJKEEEIIIQz41UhSSs1VSm1VSu1QSt0R6koJIUSwSH4JIQbqoI0kpZQZeAiYB0wFLlFKTQ11xYQQYrAkv4QQg+HPSNKxwA6t9S6tdRewGDg3tNUSQoigkPwSQgyYxY9zRgLlvf6+F5hxoDdMLkgdTJ2EECJYJL+EEAeUl57g8zV/Gkl++f/s3Xd8W9X9//HX0bC8986wnT1JIIsSRhhZkJAUaIAySikNoxQo5Qtd365f+RZa2kJZJbTQsAkzECAkjDASMiAkZG8nTry35SHb0vn9IdnYsZxItmRJ1uf5eORBfHV170dq/O655557jlJqMbDY9eNzWutrfHXsYKSUWqy1XhLoOvxNPmf/EQ6fsafCLb8gPP49hMNnBPmc/uTJ7bZjwKAOPw90betEa71Eaz1Zaz0ZGO2j+oLZ4pPv0i/I5+w/wuEzHk/yq3vh8O8hHD4jyOf0G08aSZuA4UqpPKVUBHAF8JZ/yxJCCJ+Q/BJC9NhJb7dprVuVUrcC7wNG4Cmt9Q6/VyaEEL0k+SWE6A2PxiRprd8F3vXiuP3+3ijh8RlBPmd/Eg6fsQvJr26Fw+cMh88I8jn9Rmmt+/qcQgghhBBBT5YlEUIIIYRww6eNJKXUU0qpUqXUdl8eN5gopQYppT5WSu1USu1QSt0e6Jp8TSkVqZTaqJTa6vqMfwh0Tf6klDIqpb5WSq0IdC3+opTKV0ptU0ptUUp9Geh6gpHkV/8RThkm+eXnc/vydptS6mzACjyjtR7nswMHEaVUFpCltd6slIoDvgIWaq13Brg0n1FKKSBGa21VSpmBz4HbtdbrA1yaXyil7gQmA/Fa63mBrscflFL5wGStdXmgawlWkl/9RzhlmOSXf/m0J0lr/SlQ6ctjBhutdZHWerPr73XALpyz+vYb2snq+tHs+tMvB68ppQYCFwH/DnQtIrAkv/qPcMkwyS//kzFJvaCUygVOBTYEthLfc3XhbgFKgdVa6373GV0eBO4GHIEuxM80sEop9ZVrdmkR5vpzfkHYZJjkl59JI6mHlFKxwGvAHVrr2kDX42taa7vWeiLOGYqnKqX63e0HpdQ8oFRr/VWga+kDZ2qtTwPmAj9x3VoSYaq/5xf0/wyT/Oob0kjqAdc97teA57XWrwe6Hn/SWlcDHwNzAl2LH0wHLnbd734JOE8p9VxgS/IPrfUx139LgTeAqYGtSARKOOUX9OsMk/zqA9JI8pJrQOB/gF1a678Huh5/UEqlKaUSXX+PAmYCuwNble9prX+ptR6otc7FuVzFR1rrqwNcls8ppWJcg3RRSsUAs4B++wSX6F445BeER4ZJfvUNX08B8CLwBTBSKXVUKfUjXx4/SEwHrsHZat/i+nNhoIvysSzgY6XUNzjXvlqtte63j5eGgQzgc6XUVmAj8I7WemWAawo6kl/9imRY/xHQ/JIZt4UQQggh3JDbbUIIIYQQbkgjSQghhBDCDWkkCSGEEEK4IY0kIYQQQgg3pJEkhBBCCOGGNJKEEEIIIdyQRpIQQgghhBvSSApjSqlfKaVk9WghhPCCUmqGUuroCV7/r1LqT31Zk/APaSSFMKVUvlKq1DVVe9u2G5RSazx5v9b6/7TWN/ihrjVKqSallFUpVaOU+lQpNd7X5xFC9F+ufGt05Ujbn0cCXZcIL9JICn1G4PZAF+HGrVrrWCAZWAM8G9hyhBAhaL7WOrbDn1sDXZAIL9JICn1/Be5qW8zxeEqph5RSBUqpWqXUV0qpszq89vu2VaOVUu8ppW497r1blVKXuP4+Sim1WilVqZTao5Ra5ElxWms7zhWqx3Q47lSl1BdKqWqlVJFS6hGlVITrtUeVUn87ro63lFI/c/09Wyn1mlKqTCl1SCl123HH/dL1WUuUUv12AU8hwpVS6jql1OdKqQeUUlWuHJh73OsHlVJ1rteu6vDa9UqpXa73va+UyunwmlZK3aKU2ud67/9TSg1VSq1zZcqytpzq8J5fKaXKXb1eV9ENpdQ81zp51a7jneLr70X4hzSSQt+XOHtq7urm9U3ARJw9Oi8AryilIt3s9yJwZdsPSqkxQA7wjut23mrX+9Nxrjj9mGufE3KFylXA+g6b7cDPgFTgO8D5wC2u15YCVyqlDK73pwIXAC+4tr0NbAUGuN53h1Jqtuu9DwEPaa3jgaHAspPVJ4QISdOAPTgz5C/Af5RTDPBPYK7WOg44A9gCoJRaAPwKuARIAz7DmXsdzQYmAacDdwNLgKuBQcA4OmQkkOk6/wDgB8ASpdTI4wtVSp0KPAXcCKQATwBvKaUsvfsKRF+QRlL/8Fvgp0qptONf0Fo/p7Wu0Fq3aq3/BliALr/IwBvAxA5XVlcBr2utbcA8IF9r/bTrOF8DrwHfO0FN/1RKVQN1wK3AHzrU9JXWer3rWPk4Q+Mc12sbgRqcDSBwNsjWaK1LgClAmtb6j1rrZq31QeBJ1z4ALcAwpVSq1tqqte7YMBNChJ43Xb0vbX9+7Np+WGv9pKuneimQhXO1eAAHME4pFaW1LtJa73Btvwn4s9Z6l9a6Ffg/OmcewF+01rWu92wHVmmtD2qta4D3gFOPq+9/tdY2rfUnwDuAux72xcATWusNWmu71nopYMPZEBNBThpJ/YDWejuwAvjF8a8ppe5ydS/XuBotCTivfo4/Rh3OX/K2BseVwPOuv+cA0zqGFc5GVOYJyrpNa50IROFsZL3a1sWslBqhlFqhlCpWStXiDKuONS3FefWG679t45lygOzj6vgV34bjj4ARwG6l1Cal1LwT1CeECH4LtdaJHf486dpe3LaD1rrB9ddYrXU9cDnOBlGRUuodpdQo1+s5wEMdsqMSUDh7gtqUdPh7o5ufYzv8XOU6X5vDQLabz5AD/Py43BrUzb4iyEgjqf/4HfBjOvzCu8Yf3Y3z6ibJ1WipwRkM7ryI81bXd4BI4GPX9gLgk+PCKlZrffPJitJaO7TWnwH7gVmuzY8Du4HhrltjvzqupueABUqpCcBo4M0OdRw6ro44rfWFrnPt01pfifOW4P04G2YxCCHChtb6fa31TJy9S7tx9jaDMz9uPC4/orTW63p4qqTj8mUwUOhmvwLg3uPOG621Pv5WnwhC0kjqJ7TW+4GXgds6bI4DWoEywKSU+i0Qf4LDvIvzquePwMtaa4dr+wpghFLqGqWU2fVnilJqtCe1uRpdY4C2bu84oBawuq7yOjW2tNZHcY6lehZ4TWvd6HppI1CnlLpHKRWllDIqpcYppaa4znO1UirNVXe16z0OhBBhQSmVoZRa4Gq82AAr32bAv4BfKqXGuvZNUEqdaMiAJ/6glIpwXZDOA15xs8+TwE1KqWlt46aUUhcppeJ6eW7RB6SR1L/8Eeh4ZfM+sBLYi7MruAnnVY1brvFHr+MaKN1hex3OXqArcF4pFePsqTnRwMNHlGtuE5yNnd9ord9zvXYX8H2c45WexNm4O95SYDwdpg5wjT+Yh3Mg+iGgHPg3zluIAHOAHa5zPgRc0aGBJYQIPW+rzvMkvXGS/Q3AnThzqhLnWMebAbTWb+DMrZdct/m3A3O7OY4nioEq17meB27SWu8+fiet9Zc4e/kfce2/H7iuF+cVfUhprQNdgxBdKKXOxnnbLUfLP1IhhBABID1JIugopcw4J8j8tzSQhBBCBIo0kkRQcY1zqsY56PLBAJcjhBAijMntNiGEEEIIN6QnSQghhBDCDWkkCSGEEEK4YfLHQd/a+6rcwxMijFw84rLuJigNOZJfQoSXvMRhjE+f6DbD/NJIamipP/lOQggRhCS/hAgvzXZbt6/J7TYhhBBCCDekkSSEEEII4YY0koQQQggh3PDLmCQhhPeUVsQQj8VgQRF846A1GpvDRj21aCVjm4UQ3wr2/IKeZZg0koQIEjHEEx8dDwZNUGaMBovDAg1gpSbQ1QghgkjQ5xf0KMPkdpsQQcJisAR3wCjAoJ11CiFEB0GfX9CjDJNGkhBBQqGCO2AAZ4nBXqQQoq+FRH6B1xkmjSQhRLsNazZy9XnX8f1zruX5x14MdDlCCOEVX2eYNJKEEADY7XYe/O3D/OW//8fS1f/hw7c+Jn/f4UCXJYQQHvFHhkkjSQgBwK4texiQk0324GzMEWbOmz+Dz1etDXRZQgjhEX9kmDzdJkQIuumyn1Nd3dBle2JiNP969W89OmZ5STnp2entP6dlpbFry+4e1yiEEO74I7/APxkmjSQhQlB1dQMjbnqwy/a9/7ojANUIIYTnQim/5HabEAKA1IxUSgtL238uKyojNSMlgBUJIYTn/JFh0kgSQgAwasJIjuYfo6igiJbmFj56ew3TZ54R6LKEEMIj/sgwud0mhADAZDJyxx9/yl3X/gKH3cGFi+aQNyI30GUJIYRH/JFh0kgSQrQ7/dxpnH7utECXIYQQPeLrDJNGkhAhKDEx2u0gx8TE6ABUI4QQngul/DppI0kpNRJ4ucOmIcBvtdZdh6YLIfpEbx6TDSeSX0IEn1DKr5M2krTWe4CJAEopI3AMeMPPdQkhRK9JfgkhesPbp9vOBw5orWWtAiFEqJH8EkJ4xdsxSVcAsuplkNjy+TesXLaKssJy0rJTmbNoFhPPPCXQZQkRrCS/gojklwgFHvckKaUigIuBV7p5fbFS6kul1JcfvrLGR+WJ7mz5/BtefnoZSTOjmPK70STNjOLlp5ex5fNvAl2aEEFH8iu4SH6JUOHN7ba5wGatdYm7F7XWS7TWk7XWk8//3gyfFCe6t3LZKnIXZpE0NB6D0UDS0HhyF2axctmqQJcmQth9//NXFky6jOtm3RDoUnxN8iuISH4Jf/BHfnnTSLoS6aoOGmWF5STkxnbalpAbS1lheYAqEv3B3Mtm89elfw50Gf4g+RVEJL+EP/gjvzxqJCmlYoCZwOs+PbvosbTsVGryrZ221eRbSctODVBFoj+YMO0U4hLiAl2GT0l+BR/JL+EP/sgvjwZua63rAVnpMojMWTSLl59eBgudV2A1+Vby3yzi8h8ucrt/uAySDJfP2aa6soZ/3HM/d/7lHhKSEgJdTlCS/Ao+3uYXhMfvdjh8xo5CIb9kxu0Q1faLs3LZKvYWFpCWncrlP1zk9heqbZBk7sIscnNHU5NvdQZUh+P0B+HyOTta/cq7tBbsZdWyd/nejVcGuhwhPOJNfkF4/G6Hw2c8XijklzSSQtjEM0/x6Jen4yBJwPnfhc7t/emXL1w+Z5vqyho2vbuaxy7N4pYVq5m16MKgvRoT4nie5heEx+92OHzGjkIlv7ydTFKEoJ4Mktzy+Tfcd9sD/PyyX3DfbQ+ExKO54TYYdPUr7zJ/mGJ4RiTzhylWLXs30CUJ4Rfe/m5LfgW/UMkvaSSFAW8HSYbqHCbhNBi07SrsqknOq86rJsWz6d3V1FTV9Oq4f/jpvdxyyW0cOVjAZadfwTsvv+eLcoXoFW9+tyW/gl8o5ZfcbgsD3g6SDNVu354MBg1VbVdhKbHOX+GUWFP71Vhv7u3/7uFf+6pEIXzGm99tya/gF0r5JY2kMODtIMmywnJyc0d32paQG8vewgK/19ob3n7OULZ13WY+LmzixW8KO21PLt8ctAMghegpb363Jb+CXyjllzSSwoQ3gyTbun3brsQgdLp9vfmcoexPS/8a6BKE6FOe/m5LfgW/UMovaSSJLkK52zfc5hkRQnQm+SV8SRpJootQ7fYNx3lGhBCdSX4JX5JGUpjr7solFLt9Q3XAphCiZyS/hL9JIymM9bcrl1AdsCmE8J7kl+gL0kgKY/3tyiWUB2wGi9LCUu69836qyqtQSjH/you47PpLAl2WEF1Ifonj+SO/ZDLJMNbfZnids2gW+W8WUXWgFofdQdWBWvLfLGLOolmBLi1kGE1GfvKbm3jmg6d4/I2HeePZ5eTvOxzosoToQvJLHM8f+SU9SWGsv125hOqAzWCSkp5CSnoKANGx0eQMHUxZcTm5w3MCXJkQnUl+ieP5I7+kkRQm3A1wPNGjssHyKKq3dYTigM2eWr9mI6+98BpFBcVkDcrk0u9fyukzpvrs+EUFxezbuZ8xE0f57JhC9NTxWTBi7HA2vbkpqPPLXd0nqkXyK/jySxpJYaC7AY6X/3ARl/9wUZcrFyAoBkT2t4GZvrR+zUaefGIJuQuyGZw3jupDdTz5xBIAnwRNQ30jv735D/z0t7cQExfT6+MJ0RvusmDTm5uYMnUKe1fvC8r86q5uybDQyi+PGklKqUTg38A4QAPXa62/6NWZRZ850QDHX/zzri6/rPfd9kBQDIjsbwMzfem1F14jd0E2ycMSAJz/XeDc3tuQaW1p5bc3/Z4LFp7P2XPO8kW5ASX5Ffq6y4K9q/fxi3/e1WnfYMmvE9Ud7hkWSvnlaU/SQ8BKrfVlSqkIILrXZxZ9xttHS73dv7vu5N52ecsjsd0rKihmcN64TtsS8+LYU9C7QYpaa+6/5wFyhuVw+Q2X9epYQUTyK8R5kwU9yQ3JsL4VSvl10kaSUioBOBu4zlVEM9Dsk7OLPuHtAEdv9u+uO/ngjkNs2ripV93M/W1gpi9lDcqk+lBd+5UYQPWhOrIGZfbquNu+3M6q1z9gyKg8fjT3RgB+fPf1nH7utF4dN1Akv/oHb7LA29yQDOt7oZRfnvQk5QFlwNNKqQnAV8DtWuv6Hp9V9Clv1zKas2gWzz7+HCkz4rCkGbGV2alYU8c1N1/dZd+Vy1YRNyaKPW8eoqGsiei0SFLHJLH6zQ+ZeMvIXnUzh/IaTP526fcvdd7DX+C8Aqs+VEf+8kJ+fOPiXh33lCnj+ST/Ax9VGRQkv/oBb7LAm/wCybBACKX88qSRZAJOA36qtd6glHoI+AXwvx13UkotBhYD3PC76zj/ezN8WqjouZ48WmpvclD8cQU2awuWWDMGm/t/KgV7j2Go0gxekE5sThTWw40cWV5KfXWD2zlMvOlmlkdiu9d23/61F15jT8FhsgZl8uMbF/v06ZB+QvKrH/A2CzzNL5AMC4RQyi9PGklHgaNa6w2un1/FGTKdaK2XAEsAXtqxVPusQuET3jxaunLZKkZfl9epi7jqQK3bKyiHsjNobhrxQ5zDPOKHRDNgbgrWJxp80s0cTo/Eeuv0GVODMlSCjORXP+FpFniTXyAZFiihkl8nbSRprYuVUgVKqZFa6z3A+cBO/5cm2vT1nB9lheXYNkey6eFttDTaMUcZGTQ9k9rCpi77mowmIuJMtDbZMVqM2G12IuJMWCwW8t8skm5mEVCSX4EXzPkFkmHixDx9uu2nwPOuJ0MOAj/0X0mio0DMs2G3OTi6qYQhP8gifmg0tQcaOPRiMTEqrsu+A4ZkY2wwYDc7sLU0YjKbMTZYyB2Vw5xFs6Sb2Qsa7XxAXQW6khPQrjpDi+RXgAR7foFkmK+ERH6B1xnm0dptWustWuvJWutTtNYLtdZVPS5QeKXjPBsGo4GkofHkLsxi5bJVfjtnQ1M9uYsySBgegzIpEobHkLsog4amrmNd5yyaxZG3irEWNKLtGmtBI0feKmbOolkc3HGIw/uOUFlayeF9Rzi445Dfau4PbA4bOBRB2wbRgEM56wwhkl+BE+z5BZJhvhL0+QU9yjCZcTvIBWKejRZbC4nD49B2jW51oJSBxOFx7LcVut3f3SDJte9+wTfbt5J3TRYJw2Kp2W/l/ZfeB+CSGxf4rfZQVk8tNIDFYEEF4eWYRmNz2Jx1CuGBUMgvkAzzhWDPL+hZhkkjKcgFYp6NyJhI6g41kDTq2+7pqt11RMZEdtm3u0GS6x/4ipE3Dmo/RtKoOLgCVj/7oQRMN7TSWKnBGsxXYsGZfSJIBXt+gWSYr4REfoHXGSaNpCDnq0VoX39iOavf/JCm+iYiYyKZufB8LrlxgdtjzFx4vvOK6Qrar6AOvVTE7IWzu+xfsPcYuT+e2OlcCbmx2FsdJAw77vHZYbE01RcE1eKTQgj/8eUi2u4ybMjYvF7l15xFs7rt7eouwxrrjnDfbQ9IfoUJaSQFue7m2QDPF3F8/YnlvP/O+126jYsOF3O0pMDtwrezmc3qZz+kqb6AyJhIZi+czZCxeV3OefhwKwdXHWXY3MHt56vJt2I0GajZb+10NVez34o5wiwLPgoRJnyRX+A+w1a++D7GN0xM/MnIHufXy08vw2KxuO3tcpdhlbtqUWZF0swoya8wIY2kEOBung1vFnFc/eaH5F2T1aXb+MsnvuL0uya4PcaIscO71LFy2SoyZyVjMzdSmF+LyWxm6KUD2PPcEVJGJHa6Upw8fRLfvLS1y9VcbGycLPgoRBjpbX6B+wzLWeTg4NLCXuVX5qxkyt6tdfuov9sMe7mI3JnZkl9hRBpJIcqbAZFN9U1uu43trQ63M8p++c0uDuUf7NLz5LDC6JmDsMRHEGGJwm6zY2jRGDFStbqxy2Oyrz+xvMvV3NrVX/R6FlshRGjzdkC3uwyLy42k1Wbvcgxv8svWYqOmopYf/eI6t4/6H59hxmYTI+fneVy3CH3SSApR3gyIjIyJdHvry2gyuD1Gq25m5BVdByzuevwwzXWtxAyIAsAUaaSurhVlVPzin3d1Oe8lNy7oMsBx7459suCjEGHO2wHd7jKsLr8Jk8XYab+e5FervbXbWbGPz7D7bntA8ivMSCMpRHmzcGJ3AxknT59E/psFXY6hHdp9z1OzgyNvltJ8fiuR6WaaSlso/rASg/b8n9GcRbP47z+ewWFp7fS47XU/u7bX34kQIjR4u/Cruww7vKyECJOFqgO1kl/Cb/zSSLLWWImOi8Zg8GiuStED3iyc2HYldPytr45Pt3U8xhP3/tttz5PBaKCl3k7xmkparK2YY0201NtJS0n2qnZjpIH0GSmdVugWQoQPbxd+dZdhc1yDsSW/hD8prX0/qcG9d16u95TVEhMX3b6ttqGJ5CFZKOWcpCAuJY4RU0d9W4hBEWGJ8Hkt4ai3j9i3P0lyRVannqcIh4X0+fEkj49vX+Ooclst1attpGWnenS++257gKSZUV3mJKla3ej2lp0IDVeM/UFsz8m+AAAgAElEQVS/mUFJFrgNrL7OL9s6xWU/vsSjc0p+9U/Dk0cxKWua2wzzS0/Sry87s8s2W3MLJZXftri3Hi7jy2dWt/9cWVtPY0QERpPzHnO9rYW0Edntjaqc8UNIG/jtfV/ppXLPF2slddfztHb1Fww4JRtrTV37Gkdpw1M4+MIORvxgkEfnC8QMvEKI0NDX+TXglGy+fnuPx+eU/Ao/fTYmyRJhZnDmt92agzOTmT9tZLf7N9qaKSipBkBrzeurv2Jrg3MV54q6BoiLxWBQaK3RsZEkD3A2oIwRJsafNT5sG1Ed10qCnj+i2t2g65byVjKHZrRv2782n7gBUR6fLxAz8AohQkNf51fVgVpa7a0en1PyK/wE7cDtKEsEIwant//8y5yMbvc9XFxJjbURgKIqK+//802UAWzNrdSgsERZ0A5NZFYi8emJAGQNySZ9YJp/P0QA+OpKx93stm4HWy4vYuzlwzw+n7cDNoUQ4aPP8+vNIgza6PG0JJJf4SdoG0neyOnQQ3UKMHuK+x6qbQcKabS1ALBq5UbyG5oBKK1rxJLo/CWxG2DgxKGgICk9iQFDsv1bvI/54kqnuxm6ZzOby3+4qNNAydSENCITLZ3ef6LzeTtgUwgRPvo6v9p+9vSckl/hxy8Dt1n3cMgOfCyvtrIzvxiATfllHK1tAKCotpHoxFgcWpMyPJuE9ERQiqHj8trHUQWDLZ9/w5P3/we7xU5rfQumGDNGm5Ex48ew/esdXdZuc+fmubeRND0G64EGmsqbiUyNIHZoNFVr67nx1zd0GuA4Yuxw1n66lpQZcZ2e9rjm5qslOMKIDNwWvtBdfv34nh9xcMcht+tPHs+b/JqzaBYAzz7+nGRYGOvzgduhLDUxlrMnOm8ftf23I601n209SH1ZJQ22Vj78aAtms4lqayM6NgqDUqQMzSI1J52IyIg+74k6uOMQLbqFARekEJUeQWNpMwXLy/hqw2ZG/GhgpysrwG3I1Fc3YNitGbQgndjBkViPNFGwvJT6qoYuAxzXvrSWhoomWj5u7jRviBBCeMtdfh1bUcGK/75LQemRLr1D0DXDvMmvl59expSpU7A3OSj+uEIyTHTh0b8EpVQ+UAfYgVat9WR/FhXMlFKcPXFo+8+Xnj2+yz6fbD1IRX4Rh8tq+XzFBgwGRVltAxEJMWhg2BljiI6PJjYhlsTUBJ/Wt/rNDxlyTRaxeZFEGCE6KxJt1xR/UtVlBtrVz37otpFkjDAwYG4qcXnOmWnj8qIYMDeVPf8q6DLAMWVGHC0fN3Pmbya1v7/qQK2sZSSChuRX6HCXX5b4CHY9vp/RN+d4lGHe5BcLYfVjHzLxlpFdHuuXDBPgXU/SuVrrcr9V0o+cM2FI+9/fW7+TB5ev4VBxJXmZyfzkorNoKCil1W5ne2ElVq2wNbXQYDZiibKQMjiNgWNziI6LJs41TsrdIMTubpU11TcRmxuFUWkMCowKojIisDcct8bRsFia6t0PhrRYLJiijdibHBgiFI5mjSnaCA7VZYCjJc2IzdrS+djySKwIPpJfPdDbOYug9/kVmxuFtrufRdtdhnmTXwm5sc514WQ9SdEN6VP0o/fW7+Tul95myIIMzsxNpyLfyq9ffZe/XDGfuad37oFqbmkFYN3OI+z7Ygd7iqtoNBjZsHEn2w4eJHVSPHnTM7BVNbPyrZWA+1tllmgLtfutpI2KAcBkgMaSZozRx61xtN9KZEyk27pzR+VgqGvFpprRDo0yKAx1JqLjo7oMcLSV2bHEmjsfWx6JFSLk+WLOou4GUYPn+VW114oyKrezaLvLMG/yqybfeQx5rF90x9NGkgZWKaU08ITWeokfa+o3Hly+hiELMkhz/fKlDY2HBc7tc08f02nfCLPzf4oZE4YwY8K32zNWfMDoGwaSkBdN0ZfVGLQiemAkK5a+g7m6ifjBaRjNJsbNmIDJbCI3ZyAHXjpE68VpRGVE0FjSzNG3y2htdlCysZLIjAiaSpo5uqKcOQtnu617zqJZLH30WRLPtBCbZcF6rInqz23MXHg+m97chGO+nVazDVNLJBVr6jDYTF3WT5JHYkUQkfzqAV/MWbT6zQ/JuyaLxJFxtDa3kjgyjrwT3Op3l18Fb5WRFJfIwReKGDiv5aQZ5k1+HXm7pH27PNYv3PG0kXSm1vqYUiodWK2U2q21/rTjDkqpxcBigCfuvpzFC6b7uNTQc6i4kjNz0zttS8mN5fPiox4fo67extjRcTjQZHwnAaPRSPZZyay/azdLb5hFZV0DZVVWVrz9BdbGZgp3H6OpsZmDLxWj7RpzgonWJju6FYo+qKSlvhVzjAmzMjNkbF63522saaRlTQPFjXaMUUZa6xVDxuYxZGwezz/0AtXFFSRmpnDN7VcD8kisCGpe5dcNv7uO8783IwBlBhd3cxbFDYxmy669rHv9c4+OYa2sp7EonobCJrTDgTIYUEphraxvP8bIM8aQ4prGpbHair3JwbFVFbQ22jFFOW+bmaLArMweZ5in+XXV7d9n4pmnuF0DTjJMgIeNJK31Mdd/S5VSbwBTgU+P22cJ4LxCC+EpAHwpLzOZinxre08SQEW+lbxMzxdUjIuxULO/nughFiJMima7nbqDNuJiLJhMRtKT4khPimPskCwANm7fSeSsaKKTLDjsDo5sqeLwnnJq9tQzcFwaWimis6IwRhp57+X3mTB9fPvSL21WvPAeORenMP6UGEwGRatDs+2bet554T1+eu8tZJgUr96Uw09W1DN03BDiEmMlUETQ8ja/Xtj2tHY4HH1eZzBQSrXngbs5i4q+LCcjJpK7xg706HgvxEUSEWUgcrCFCCM026HpiI2EOOcxquoaeWz1Zs695gIAojISmHZ1ZpdB1Fse28PEmzwbXO1tfoHz9uGE6ePpOCWOp/8G6qqs2O12ktOTPNpfhJaTNpKUUjGAQWtd5/r7LOCPfq+sH7hjwQzufultWODsQarIt3JweQl/uWK+x8e4de5Z/PXFjxlyeQbJw6Kp29/AwZdL+J+557rdv633ymB0LssydmYUjiFQd6Ce6iO1VB2rJzY5kriUSCq21fHOn18kOiGWVrOJ1LwM8iYO5diBYwwZGc8nS45SX9FCTIqZ9FPjOXjgGGvf+ISLhxkYnmHh4mGNfP76GuZeP4/aqjqe/u2TXP//FrcPOBci0HqSX1sfXd4ntQUbjaYyNpoLfjgHcD+7dOFH5Vw9fWKnJaZO5Pb55/DXd535FTcsmvr9DRx+t4z/mX8ugzOTiY9pgJ3f9qy39V5prSndUoG92YF2aOqrGijcUMqWp3bTVGUjMslC+thkyvfUsHn1V+3vT8xIcptf8XlRHNxcxNP/8zgJLVaWb24loaWJp+56jPGnOhdaryyt4psvdzB7xiSiIz1fbN3RaqckMoILb77Y4/eI0OFJT1IG8Ibr6sIEvKC1XunXqvqJtnFHDy5fw+fFR8nLTHYN2h5zknd+6+aFZ/L0W59Q+N9C9tocxFoMpGLmlu+e5XZ/t71XX9aiogykz04kJzudxsJmjrxRSm5WEs/9ZB4AReU1lFTW8fqrn9JY2cD2VxtIHB1DyjlJ2G0O9n9WiaNJs2P1Wn5zufPYV06K5cqX13LmJTNY+8YnGArz2xtNQgQJr/Pr/mvP74u6gtIfXvqUTf98vf3nU5MHsvmJA9TUNZAQF83UkUO4Zf53PD7eyfIrLjqS1LqG9nOam+zseHIP0UkRZA6LJybNQuVRKxaTkdLtZWSel0xkmpmmshaKPyojMyqWq5Oj28+3ac+Rb/NrbCzxE+NQZsWhT6swGSCqroKXr8wgNdZEubWVRcsq+PP8KaQkxPD351dRGdFIXoyRO6/y/N+Aw+Hgdy92/t4Adh4s4ru/u5b4pLhu3ilCgcy4HeT+/vwqOPYVd5797XxKf/+0BgZM4s6rZnXZv+MTdW29Vx8/vJNBl6aTMDYGi0Vhs2nqdjdg+Uzz5aN3UV5t5cb7nmPJL68hJSGGyT95ANtZiuRxcTQW2Wgsb6Z4QzVNe2ycmmHi1IEWNubX8/8WZrEuv5kdmaex75ON/Gmqjd9siuTmJ37T3pskPUzhoT/NuC355Tu+yK+Dy0uw1bRgmR1F3KjoLhm28t6b3OZXyvh4WutaaShr5thH5UQUKX52WhTXnhbDja8UsWRRNku/ssKASVx70Rlc+vMH+fW0Zu7dGMHrf/sZKQnOJ+yOz0dPPfruJpgxkZSslN5/kcKvZMbtELZm814KS228sK200/bskr1uQ8Zd71VCZCSZeQnUVzfSoEEpSB8cT0FDCQDPvLOOquIClq5Yy51XzaK2oYnx4/Ior7NisBhIzo1lxPhMPvjVVhrsEbz5TT0W3czlS45ijjBhs3/AmGTNsUoDp8TYO/UmSQ+TEOHLF/n1lyvmc/sTr5M+ON5thnWbX7VWWlscxKVGcuaPRvHBr7bywjYDj6yrJtHUzJSHj5IcF0V2yV4AzhnQTE6cnXOym9uPBV3z0VMGYO/2g1QUV3q0f1JGUpdpB7TWHNp5mNbmlm7eJXwhalgck7LcvyaNpCD31t9u9fo9c08f0+mW3nn/8zAV+bWMHhVBlFnR2KLZtbuWgakJlFdbWfHJJh6/JJWbV2ziB/Omk5eZTGtZC8OHprUfo+xALaePzeX5e65l0d0P8fi8aG5e0cC/fvNjfvyHJdx7RgM1NhhY3cJ/n3kP6+EKzHHRbPxwLffPTeT+1c7bctKbJET48EV+Afz1tQ/dZlhGUlyv8uuVv96B1ppLf/4g959pJyfJxIV5rdzz8UZ+MG86Wusux/e0N+mqc8axbvthdJlnjaRn39vExb/6fqdtNRU15L/2Cd89fZRHxxA9k5Nu7fY1aSSFgWHJSbyzchsD4tKwDLJQe8xG2coypo4ezzPvrGPuEDA21zJ3iJmlK9a2Dzi3z9fURdiIa7Zw+O1S/nLFfJ55Zx3zhhkYmW5h3rAm7nnkFc4Z0MywFBOZsUYmZZswmCExwxkkabkmisqaSWyp57FbH2T86eMZMC6XgSMHSje0EMIj3WVYtCWFi4f2PL+WrlgLOHuRcpOMRJoM5CYZ23uTgC77e9qbFB8TxZxpnjdu3t5b0mWbw6EZN3QAc6eNdvMO4TOZOd2+JI2kMFBQXEFzsY11SwpptmsijAqTzcHBuFL2HzrMwzPB3tLMhUNN/HS182rpL1fM584lr3OstIoB6Un8ffElTBk1mL8+9QbLFjkHIl57Wgz/evgQG2ytrNhpxGAAhwPKGhyMrNiOo6WRZYsSSY018f2JsSxaVsc/Lp7K+j1H2fTWOnY226mobyIqKY7cScPIGpotgxyFEF24zbAmB0fMJVw4K7nH+bVo2SaUOYq9h+pYsdPQTYZ13t+b3iQR+gyBLiAclFdbufQX/6Kipr5H739v/U5m//IxRvzwT8z+5WO8t36nV++fOXU0d5+XxhdXxfD1NdF8cVUMd5+XTlyMhblDoM7awD0fNFFnbWDuEMXSFWuZMmowKdrBc7MjSNEOpo7Oab8KS411tq1TY01cMSGaaTkxrLoph5ykCFbfnMvPZjiPffy+84YZWLpiLRdNHcmtcyZRvHMv/7z8LB7/7umMOFbGgec+4LNHlrP8gVf47OU1VBRXYm+1n+ijCSH8LND5Be4z7MbTY5k80NKr/Jo3zEB8jIWfzUj3KsN6+52I0CE9SX2gpwP/wP36b3e/9DaAx1MJrNm8l92H6njk0yYcWmNQCktkC832WnYY4cGGJjJi4NKX64mJtjOqzDmQ8TvpjQxLdv536Yq1bgdhllbV02KHqY8c6zQYsry2gIKiyG4HbLZ9J8+99wV3XjWLK2ecwpWufZpsLRwsrGDV6q/4sLASh8GAOS2BweNzGThqEFExUV59h0KIngt0foH7DGvVimY7XHpE9zi/gPasOn5A94kyDOjxdyJCizSS/MzdwGhvumq9Wf+tO0/973UsvPMfGG2tLJkfzeK3G3BYYvjP72/k+t8/gTGq8/YHbl/E9b/7F/93hiY30cT8Ya386oP1LH/w525rL6+2dhkMeaLPeLLvJNJiZkxeJmPyMgHnEx7l1Vbe33KIrz79hnqDkZboCOLTEhl/7gSiY6PbJ88UQvhOMOQXuM8wmzEOg0FhbqntVX61fU5PM6x93x5+JyK0yP+z+FnngYKG9sGA4Fk39qHiSlJyOz8RlpIbyyHXY6WeHOOZd9aRZm5iwUgzI1ONLBhpJtXsHHTd3fbvpDcyNMmAQWmGJhnar8bcnfNEn9Hb78QdpRRpSXFcfe4p/OP6WSy57nzuu2ACi3NSOfrSx7z/l5dZ8bdXWf/2OiqKK/HL3F9ChKFgyK+2Oo7PKlujlQRl7XV+nexzevOd9EZNVS37Nu/zeDkU0TekkeRHbVdh157mvMq49rQYVnyyqVPjoq3LtjttM2h31HH9N0+O8f6GXWwtbGTaQMXOshamDVRsLWxk447D3WzP57ktDVz4fAPT/l3Phc838NyWBt5dt73LOU/2Gb39TjyVkRzP2CFZ/P6Kc1h6y0X8+5pzuSknjfK31rHq/15k5T9eY93ydRzbf8yr4wohnIIlv8B9hh2psrGzpLlX+eXJ5/TmO+mNh687n9zDRaz72yus/PtrfPDUSrZ/vl0u+gLM+Pvf/973Ry3Y6IeDhp5/vbaGEeZizh/unDY/OsJARV0zW4tbGT44g3v//TqPL0jgH6sPM+9s9+sFpcbGsOy1zVjSzUQlmCk/VMfB5SX872WzSYqN8ugYZZW1TE2q4fLJ6aQlxTE4LR47RuyxGXxvtImzhydxyxsV/OjMAURZIjhQZ+LykQ7+dJ6Fu86wMGuYCaPBQI0phQumjOl0zvLaJsZFlbn9jN85ZahX34m7/T1lMhlJTYxlxrgcFk4dwYKJQxis7Rz6aj/r39nA/s37OXa4hLjUeKJjZUyTr41Ln/iHQNfgM5JfQPDkF7jPsPWHbcwaEcWssWk9zq95Z0/imXfWeZxJ/sovgChLBBPyMrlw0nAWnjaUU1NiiamrZ/LwbNJlfjn/is2ApFy3GSbLkvjRxT9/hMLS8i7bs9NTmXHaiPbp+k80TT84Bz8+uHwNh4oryctM5o4FM5h7+phOU/6f6Bjd1VFe20RqfCRFFXWkRdopazKSlRJHfnENZoMm2qyJjVBYmzUNLQpLZCS3LTqv0zmf2dqMydD1f+7s9FS3E8md6DvpycRz3th5qIiX1+/lUEUthrgYkoZnM2jUILLzuplqVXhMliXpf4Ilv7qr5Vh5HWYj2B30OL8YMMk1oNuzTApkfgk/yjwFhpzjNsOkkRQAbQP/li2K67DQYt1JBzz7+hgAew6XcNFP7+f1RdFcsqyB9x79BUlx0W6P/a/f/Jib/vRkr88ZDLTWbNiRz+f7ithbWkOdwUDeGaMZOGwAyelJgS4v5EgjKXxIfol+5wSNJLndFgC+6LI92THKq61c8/unuGDK6G67sAF+/H9LOS+jjikDDLQ64IX1x6iubWCEuZgx6Wa+93Q+C8YnYGtu5T9rDjBzYLNfupr7mlKKgelJnDF6EPMnD2fBhDz0kVJ2rtvJF6u+4uA3h1BRZixREURYuv/+hJPcbgsfkl+i35HbbcHFF122JzvG359fxYrVnzBv5jnddmG3XYW9uSgSswFaHLBwWRM5gwZSU1vb5TZc2+253tQdKmrrG1n99QE+21dEYYONuMxkTrngNDIGpwe6tKAkPUnhQ/JL9Dtyuy28lFdbufTnD/Lrac3cuzGC1//2M1ISYiivtnLjfc+x5JfXkJIQw3fvfpSxxsNcc4qJnAQDh2scPPtNKzvsOdz3k8u46Kf388gcC7eutPHeo79g+KDwbSAUlFTx4tqdHKptpMLWwsjpYxl66nAioy2BLi0oSCNJ+Ep3+dX2WluG3XDvfyW/hG9IIym8/P35VVTv+ZyrRtt5fpeRxJFncudVs7pcneUu+AW2pia3AxxPHTmIYTqfG0418++vW9ivcnnjLz8J9EcLCi2tdt7esIcPdxVQ02InLiedCTNPIzE1MdClBYw0koSvdJdfba+1Zdg/l30k+SV84wSNJI/nSVJKGZVSXyulVviuMuFr5dVWln+0kQvz7OQkmbgwz87yjzey90hp+8y5bfN6fLn0N4zOSWPDHUPZevcINtwxlNE5abx0741s232AS0aZyEk0cMkoE9t2H2BfQenJCwgDZpORS6aP4dEbZvPMjXO4fewgCpd9ykcPvsGKf77Bga/309rSGugyRQeSX6Ghu/yqqKnvNPv3ik82serhOyW/hN95M5nk7cAufxXSn3U3q6w3iyR6eoxn3lnHOQOayU0yEmkykJtk5JzsZu555BXmDgFjc237IpDdLfh48/3P8t2RRvKSDESaFHlJBr470sjdD7/io2+k/zAYDIzKyeCPV81gyQ/P59FLz2DAoULW/P1V3vnnG+zeuJtmW3OgyxSSXz0WDPnVllcdM+yeR16R/BJ+59HabUqpgcBFwL3AnX6tqB/qboFIbxaO9PQYqzfuYvu+OlbsNGAwgMMBxVY7sZYG/nR1EvaWZi4cauKnqzcRERlLeVXXBR+PFFfzXKlm5f4WDAocGsobNOaII779YvqhhNgorjt/ItedP5EaayPvbz7AF49to9RmJ2lYNqfOmkRMvDxu3Jckv3on0PlV1uBgRPl2dGsjD8+kPcP+vT6fQ0ejeGGbrdO5JL+EL3m6wO2DwN1AnB9r6Ze6WyDSm4Uj27qgf32GkXs/3njCY8ycOpqZAxq4ckI0P3zhCEuvGsxNr5UyNk1hsjeRk2TicHUTc4dEYRw02m24dZzkrX2ba+I14bmE2CgWnT2ORWePQ2vNjkNFLH16JccamonPy+SU8ybKnEx9Q/Krh4Ihv57f0sBn5RbOSG7slGE3nJ6EcdDkLhkm+SV86aSNJKXUPKBUa/2VUmrGCfZbDCwGeOLuy1m8YLrPigxlnRdDbGq/Yupue3fHOGdAMzlx9vau5+6O4Zw91sbf1pSTFmnntAeP4NCw8bCDZd8o4iMVtU0aTC2MKtvr9pxtxzi+hym7xP3+4uSUUowbks1fh2QDsPtwCS+89hmfV1mJG5zOmLNPkekF/EDyq3eCIb+cj+8XsMMIT65rPmmGSX4JXzrp021KqT8D1wCtQCQQD7yutb662zfJ0yFA97PKejPza9vjsPef2cjELBNbilq5Z20UT/52cbfHKK+2dpqF9vk/38YvH3peZpoNUvsKSnl1wz62l1QRPSCVUWeOY8DQ7ECX5ZVgfbpN8qvngiW/TjSLtmSY6KlGWzPrt+ezu6iW5GFTuPz6W3s/BYDrSuwurfW8E+4oIQN03+37WXk8Z6XWuu0OvvaiMzrNZdT2OOwtp0FmrJFiq53HNsO2poxuj/HZ1n2MNx3mj+fF8NuP6nm7IJZrxpvd7itXVsHlUGEFb2zay8ajFaSPyWH09LEkpQX/1ALB2kjqSPLLOz3JrzuvmtVpLqOlK9b2Or+2teZw1oTh3d5CkwwTnthzuIRN+4r5+kgtpbU2VHQSyePPJT4plTlnT+OMYanSSOprJ1tY9nhtC0d2nMto7u0PsX1fPmnRnQcyGkwRpCV0Xc0+MT6e/IKjfHZ9LFmxRoqsdk5fUktCfBwRZmOX88lMs8Hry11HeGPzQQ5WW0keNoAp86YRGd31300wkEZS/9OT/Dp+tmznQOze5ddZT1nbZ9Hu7pxCdHSkuJItB0v4dFcp5dYWTHEpqPhMUoZNZOi4yV32H5sd75tGksckZHqkvNrKwjv/QYyup0HF8OY/7mTpirVeXUF99+5H26/C2rRdjclkaqFr64EinlyzjZKmFoZMGcHwqaOIDaLbDKHQSPKY5FePHZ9hs6afSkzFNskv4TelVXWUVVl5ff0BDpbWY05Iw25JJClvPCMmn4NSJ4+mEzWSPH26TfSBZ95ZR5q5iZr6FlJjOg9k9HQQ4td7CtjY3MJ/vq7utN0cUeDX2oV/TRiaxSNDswD4YPN+lv/7XcqVYsy5E8gdm4s5whzgCoXommGvffQVJoOW/BI+UWNtpKKmnrc3HaKw0kpJowkdk0L8wOEMmnEhZ6T7fiyn9CQFibYrMKOthiXzo1n8dgMOSwJv/uNOGZgo3GqytbBs7Q5WfpNP1KB0Js2bFrApBaQnSUiGCV/SWlNcUcv7X+dTUGblQGUrDks8cQOGkTHiVFKzczFH+GbtTOlJCgFtV2Bn5ZoZmWpkwUgznx078aO1vnL8wrciNERazFx73kSuOXcCReW1/P3Fj1jb1MrQs8cz7LRhRFgiAl2iCCOByjDJr/4lv7iSu/+7jui4BEzpQ8kcOYe0KcM5IybWo1tnvubNsiTCj97fsIuthY1MG6jYWdbCtIGKrYWNrNrg/5UUOs56K0KPUorstAQeuG4m/71hFuMrqvnswddZueQdju0/FujyRJgIVIZJfvUvGsWg5EialZnErFwGjZpIdGxcQBpIII2kHvFmzSJPzZ42mlvPSmP6mIGMyctm+piB3HpWGrOmjfZrLccvGunLzyT6ntlk5HtnjePJmy7kvgsmYPt4C2/d9xIb3lmPX26ti5AULBkm+SWOl5eZxN8WX8CS605lhnk7Xzx+F2uf/xt11ZUByTBpJPWAP65c1mzeywvbbEx+tLT9zwvbbKzZvNevtXSe9dYgV2P9SGZKPL9adBbP3zSXOZEmPvzzC6x8/G2qyqpP/mbRrwVLhkl+ie7ERluYN20E/739fO69MJPKd+5j1YM/49D2L/u0DhmT5CVv1izyRk/m+uhtLW3vX7bIuaTVtafFsGiZ7z6TCA5KKWZPHsHsySMorqjl0Vc+5cOyasbOncroqaMCXZ7oY8GSYZJfwlMD0hL50w/Oobmllec+ep9P1zyLacA4Js69Gktk1/m2fEl6krwUTFcuva2l7WH/ArgAABa8SURBVP2psc62cmqsKeCfSfhXZko8/++qGbx063zyCkp4997n+ejZD2iwNga6NNFHgiXDJL+EtyLMJq6fPZH/3jGL20+Dfc//mlWP/pLiIwf9dk7pSfJCMF25+KIWWQgyfJlMRq674FSuu+BUdh8u4V9PvsPRBhvTFp3DoOEDA12e8JNgyTDJL9Fb44dm89DQbGrrG1ny3tN8+paV+NFnccqMBRiMxpMfwEMyT5IXulvLqKfrB7l7dNXTx1l9XYsQDU3NPPrul2w4Vs6Y2ZMZMXEYJrNn11EyT1Jo8GVudJdVnmSY5Jfwh9Vf7eepj/eTPOYshkw+j/jkVI/eJ/Mk+Yivr1w6Dlpse7+7bX1RixDRkRH8zyVn0NzSyvJ1O3l1xQYyJgxl8oVTiYiUOZf6A1/mRndZ5UmGSX4Jf5g5aRgzJw1jw64C3nz3r2ywWph86S0kpWX2+JjSkxQg5dVWFt39EI/Pi+bmFQ288tc70Fp32SYDEEUgrd9dwMMrvyJh1GCmXfwdLFHuZ7iVnqTw4i6/UhJiut0uRCBU1NTzj7e+Zl+NiYkLbiQ1y/1QghP1JMnA7QBxN2gxWAZUCtHm9FGDeP6OhVw/JJ1P/vEaHz27mqYGW6DLEgHWXVZJholgkpIQw5+uOZMl102gYc3DfLzkt5R4OchbGkkB0DZo8drTnFdY154Ww/KPNvLGhxs6bZPJ0USwmDxiIE/9ZB63jB3MuoffYNV/3qOpoSnQZYkAcJdfKz7ZxN4jpW63S4aJQEuIjeL/XXM2S66bSOu6J1iz5H8pPLTHo/fKmKQAcPfo6jkDmtlWYic1NqV9W9uVmNyjF8Fi4tAslgzNYld+CQ8+shxbYixnf/+8QJcl+lB3j97f88gr3T6SLxnWO80trVRbG1m+fj8OmTm/V7KSY4mPaubJP95IWt4YrvntYyfcX8YkBcDFP3+EwtLyTttKq+poscOA1LhO27PTU3s00aQQfeHgsXLuf3sjTzzznoxJChPu8gugvLaJ1PjILtslw7yjtaapuYXXP9/NkbI6jloVFc0mohNTyZ12IRGWrt+x6BlLVDSxCUknHJN00kaSUioS+BSw4Ox5elVr/bsTvklCRojwcsZPg7KRJPklQsGm3UfZsKeIPWXNVNY3Y4xPZ9CUOcQlppCSOSDQ5fV7vZ0CwAacp7W2KqXMwOdKqfe01ut9WqU4IU/nTxJCdCL5FQQkv761M7+YL/cVsym/lqr6FoxxacRmDSFj7Pmckjss0OWJ45y0kaSdXU1W149m1x+50upjns6fJIT4luRXcAjX/DpcXMmBY+V8tLOUo5VNqLg0zImZpI2Yz/gZ41EqKDtgRQceDdxWShmBr4BhwKNa6w1+rUp04q8FKYUIB5JfgRUu+VVSWcvBwgo+3l7E/tJ6iElFRSeSNPwMMmcNISclLdAlih7wqJGktbYDE5VSicAbSqlxWuvtHfdRSi0GFgM8cfflLF4w3efFhqvOc480hd3VmBC9IfkVWP0xv7TWfLO/kDU7CimoaKDIqjEnZpA8Ygpp0xfwnaxBgS5R+IhXUwBorauVUh8Dc4Dtx722BFgCyMBHHwqWBSmFCHWSX32vv+XX8i/28erGAoyRMUQNGk/WuHMYmTuSkYEuTPjNSSeTVEqlua7AUEpFATOB3f4uTDh1NyeJzGQrxMlJfgVWf8uvD3eVc/ZP/sb0H/2R02Z9j6xcaR71d570JGUBS1339Q3AMq31Cv+WJdrIQpBC9IrkVwD1t/ySgdbhx5On274BTu2DWoQbMgmbED0n+RVYkl8i1MnabUIIr7W22mm0NdNoa2bL/qJAlyOEEH4ha7cJITrpOAv/xt3H2HusEq01X+VXgSkKgDJrM5FJWQBExCXzr4BUKoQQ/iWNJCHCUKOtmXc37MOhwe5w8PneckwRkWgNRbUtRCY4F1qOy8glc9RMAIZNSycyOjaQZQshRJ+SRpIQ/cy2g4UcLqkFYPexao7VOQCormvEHp2KwaBwaEX2pFlERDp7hsZMHYLZYgFgXGDKFkKIoCONJCFCRJOthS+2HwKgpdXOe9tKMBjNaK0pqrMTGZcIQETSAJKHng9AzOQExgzMC1jNQvS1/33ucxpa/PMUWnGtQy4iwow0koQIMIfDwZe7C7A7HGiteWfzUWwOZ8hX1DZiiEsHnA2j5PHnYjJHgBGGXnEKFldP0OiAVS9EcCmwxTH9B7/yy7FH+eWoIphJI0kIP2m0NbP7cEn7z6u2HqPC2gxASW0z5njnuJ/WllZi8k7DEpsAwMCLvk9cYkrfFyxEfyBzGQkfkkaSEF46XFxJQ5OzsZNfWsvH2wtRSrluezmIinfe9mpqtpM4fAoGgxGAjGlzGZ6dA8DwwJQuhBDCC9JIEgIoq6qj3tXwcTg0r67bR73NDkBJdQM6Orm9IdQSlUZMWjYA5sgcRl1zR/tMvNL4EeGmrKqOytoGlq3dT0l1A80RiRiMgZmCT2tN2tDJATm36J+kkST6Ja01ZdVWHA7nnD+VtfW89eXh9tdLquppMsahDAa01jQYYolNG9D++sDTF5ORkgFApsGAyWTu2w8gRBBqbmmltKqOV9fup6jKSpU9Cps5gdi0AQy54E7GxifI74roV6SRJEJGbX1je6NHa3h7wz6qG1oAaLXb2VHUSFR0NAC25hZa4wZgiXKtNG6MZMS5d2A0OgN8qMmEJSq67z+EECHC4XBQW9/El/uK+WJ3IQVWRUOLJjZ7BANP/QEDk9MYJvNmiX5OGkkiIFpa7djtjvafv9pfzM4j5e0/F1ZaKbOZUcrZbW9rbqbRkkKkq9GjtSZz9GwSMwe3v2dqUgoGg6y0I4S3tNbYmlv55lAp63YdZWexjcZWjSUtl7jMPAbOvJIpSamBLlOIPieNJOFTDoeD1z7fha3Z3r4tv6yO0kZj+0MnWkNpvYPopLT2fWLSBpI9bl77z9EREUxISe+zuoUIN3sKyti0p4gth6sob3CgE7KJTc1m0JQbmZyRLSveC4E0kkQ38osr+XJv54VL9xTWUNbYOThLa5swxna+wsyeeA4xA75t4FjGRDE+Pdt/xQohTuhoaRXrdxey9UgVRTWtNJgTiEnKYNCp32PgpDRGxiUEukQhgpI0kvqh0qo6Nu051mV7fmkd+yuaUXzb0LG1tFDZGtk+KWEbQ1Q8AyZc1OlqMnpIAmMzB3Tab6yPaxdC9I7Wmo8272fj/nJKGxwU1bYSlTKAgaddRMLIdAanpJ38IEIIQBpJAVdQUsWBY+Xdvv7xzhKsLe5fK65tISIuuct2bbSQOeHcLt3lllHRjBkyslf1CiGCy8adh1m/r5TDlTaKalswxySSOHwKmWeMYUzWIMYEukAhQthJG0lKqUHAM0AGoIElWuuH/F2Yv1XVNrDnSMnJd3Rj08FyDlc0nXQ/u91BSaOBqJi4bvfRljiSR5ze7esZF+T9//buPUausozj+O+Z2W0xbKHC9kqLhYggGiNIKhFDjPECWBHjBRqK4oUNYA0EEuRiJChG5Y+GS1FTLjaKRWLQBCteSCQhJApawAtFAwGk9+nu0suivczM4x87BGfPO90zO+fsmTnn+0k27Lw7+857NuWX57znPe/RosF5wZ+xRT5waHnLr40vbdOO0b361dNbtfO1uvoHZmvmwpM1/+QP6aTj3kYmAAmLM5NUlXS1uz9lZrMkbTCzR9x9Y6tfuORHTyc2wLTsr5V09EmnT2kL+9knLdSJMWdkOIsDMpWb/HKX7M2LNWv+qXrr8sv19pkzsx4SkHuTFknuvk3Stsb3e83sOUnHSGoZMksv+npiAwSAqSK/AHSirTVJZrZE0imSnkhjMMi/76xcrrGxvZH2gYFZum71/V3bN3of+YVOkV/FE7tIMrMBSQ9KutLd9wR+PiRpSJJWXH2zzjx3eWKDRH6Mje3V8V++I9L+4t1f7eq+0dvILySB/CqeWEWSmfVrPGB+6u6/CL3H3ddIWiNJdz32oic2QgDoAPkFYKomfYaDjd9Hfo+k59x9VfpDAoBkkF8AOhHnQVdnSLpI0gfN7JnG1zkpjwsAkkB+AZiyOHe3PS6Jh/igLa0WIY5Wtmnmy89H2nePtN5QM67dI8PaEuh7tLJNN1y8LNLOgsj8I78wVaEMG61sV98LG1Xu629qJ7/yix23kYpWixBHvvVJjayPXvXwerXjz/R6Ndx3rcaCSABtCWXYnjtXqvLzmyJPOiC/8osiCdOq1D9D71r5g0h7Ev/Dz54zPxgmG757fsd9A8Ccj12hkfWrIhlGfuUXRRIiWl0q2zu6U7OOan445u6RYXm9qtlz5je179q5Pdh3/eB+PXXrlyPt1T07O55S3rVzu/62+rLAZx6I9fsAel87+XWo9lCGVdbfqtrekUiGkV/5RZGEiFaXyjZ89/xI+5aXn9fI+lWR9pZnP6U+Lfzi6kjzpjtWdDyl7FbSwotvjbS/dPuFsfsA0Nvaya/J2ify6n7NPf9mzRg8tqmd/MoviiTEVjsQngWqvfZq7D68dlCb7/xc6CfBRYuhBZGXffTd8nI52kO1qv9WXgl0HX/bm6R2vWX3XKC7tMyvA/tj91HdM6zKA6HH1sTPL6lFhrm0ae1VGjznygnt7W3blUT2kF9voEhCbFaOzgLVD+7X1jWXRN5bslLwLMrK/TrmsrWR9s13fk4zJ5ydSVLd65E2L5e1eOV9kfZNd6xQ3+x5oZEHxzIwMCvSltSut+yeC3SXUH5J47kREsowK/d1nF9SOMMODL+i4YduCWRY/PySkske8usNFElIxZFHD+rba9dH2i89+z2yUpztuaamf0b0yeilcjk4FgBoJZRhaeeXFM0w8itbFEkF1mpKdXjrJu0KTEt7LXqba+XnN8rr9ciCw/2vhvf28FpV9YPh6e0D+/c1va7urqheq+nyZUsndDL9T43YtXM7e5UAXSSJ/Hq9PbRgulWGbV93reZ+5qZI+8T8kqR6rRrNL2naM+zZu6/WvuFK5HjIr8lRJBVYy72Mvv0pDZ57TaR9x/3X68Bw85qf+r4xzfvsN1Xub95cbdu661v0/WlZuT/S7rWqdqz7WlNbbWxUfUfO0+KLm/cOeem2VgsZXVvXXhlptRZT3u1wKzH9DHSRJPJLGr+EdvSyqyLtoQzre36jRn5zeyTDQvk13l7TW65YF2kPZpiZaq/timRYEvlV2/cfzb/gZh2z5ISmdvJrchRJGejdRXEeuaujvm/8OGrV5rM0P8SZUqvp6trYaHMf9bqkds64LLgH09PfW84sEJCQPOXX6+0T80s6RIZZOMMm5le7+o6YJyuVIhlGfmWLIikD3b8ozlqEiWnT6gmLHF3qO2qhSv2Hxezbg3e3WbmshUN3NbVVd+/Q8EO3RHuoVaPjkKTaweDf0OvV2H/vgYFZwfaStbcOoVU/rRZbAr0iV/n1+vvnHherZ5erumc4kmGh/JKkLT/8UrifUIb5+N2/E/+O7eSXFM6eg2OjKgfuCG6F/HoDRRLa8sOHNzS9vvSc97RRIEky05Kv/iTS/NJtF6rUH110HVKeMVPfX/9k7I8MnYW10urMrJ0+DtUPgOxMzC9pPMPiMiup74hBLf7CbU3t7eSX1F6GJZE9N1y8TPMXH99RH0VFkYTOeO/uCLt7ZJhpbKDoejTDyK/pQZFUYK2mVL1W1dZ7V0Z/IfAQR69XtWNddJFkaNpYkqxW079XhzaTVOCut/GFjBPf31dK5qHuda93fBmuiNPPQDdIIr+k9jJstLJN3iLDwnftevC9SWQY+TU9KJIKrNXZxsqPn66550Xv1Kg88I1Im5X7tODz0a30N61e0dbeHpcvW6oZM5sv282Ye6xK5XJbl9bSwpkZ0F2SyC8pmQwL5Zcklcp95FePo0jKQLdX9V6vamT9qmB7RK0WPmur1dr6zL6SpXbG1ervncSttUDR5Cq/pEQyjPzKr0mLJDO7V9IySRV3f2f6Q8q/bq/qZ8+ZH3sa9+gFixK50+X2h55o6/3tSGoxNnoTGZasPOWXlEyGkV/5FWcmaa2k1ZJ+nO5QgGa9ux8LusxakWHIABnW+yYtktz9MTNbkv5Q0IvSnHpPez+Wbr9sgGSQYTiUXs0w8mt6sCYJHenls6FeHjuAZPRqDvTquHtNYkWSmQ1JGpKkFVffrDPPXZ5U15hmnKGgaMiv/CC/kKTEiiR3XyNpjSTd9diL0/+YdiSGMxQUDfmVH+QXktTeA6kAAAAKIs4WAPdL+oCkQTPbLOlGd78n7YEBTJsjCWQYskKG9T5zT35mmelqoFguOfP4ZJ4V0wXIL6BY3rHwCL3vrYPBDONyGwAAQABFEgAAQABFEgAAQABFEgAAQABFEgAAQABFEgAAQABFEgAAQABFEgAAQABFEgAAQABFEgAAQABFEgAAQABFEgAAQABFEgAAQABFEgAAQABFEgAAQABFEgAAQECsIsnMzjKzf5nZC2Z2bdqDAoCkkF8ApmrSIsnMypLulHS2pJMlLTezk9MeGAB0ivwC0Ik4M0lLJb3g7i+6+wFJP5P0iXSHBQCJIL8ATFlfjPccI2nT/73eLOm9h/qFwVkzOhkTACSF/AJwSIfPbF0KxSmSYjGzIUlDjZf3uftFSfXdjcxsyN3XZD2OtHGc+VGEY5yqouWXVIx/D0U4RonjTFOcy21bJC3+v9eLGm1N3H2Nu5/m7qdJentC4+tmQ5O/JRc4zvwowjFORH61VoR/D0U4RonjTE2cIunPkk4ws+PMbIakCyQ9lO6wACAR5BeAKZv0cpu7V81spaTfSSpLutfdn019ZADQIfILQCdirUly94clPdxGv7m/NqpiHKPEceZJEY4xgvxqqQjHWYRjlDjO1Ji7T/dnAgAAdD0eSwIAABCQaJFkZveaWcXM/pFkv93EzBab2aNmttHMnjWzK7IeU9LM7DAze9LM/to4xpuyHlOazKxsZk+b2fqsx5IWM3vZzP5uZs+Y2V+yHk83Ir/yo0gZRn6l/NlJXm4zszMljUn6sbu/M7GOu4iZLZC0wN2fMrNZkjZIOs/dN2Y8tMSYmUk63N3HzKxf0uOSrnD3P2U8tFSY2VWSTpN0hLsvy3o8aTCzlyWd5u7DWY+lW5Ff+VGkDCO/0pXoTJK7PyZpNMk+u427b3P3pxrf75X0nMZ39c0NHzfWeNnf+Mrl4jUzWyTpY5LuznosyBb5lR9FyTDyK32sSeqAmS2RdIqkJ7IdSfIaU7jPSKpIesTdc3eMDbdKukZSPeuBpMwl/d7MNjR2l0bB5Tm/pMJkGPmVMoqkKTKzAUkPSrrS3fdkPZ6kuXvN3d+t8R2Kl5pZ7i4/mNkySRV335D1WKbB+939VElnS/pK49ISCirv+SXlP8PIr+lBkTQFjWvcD0r6qbv/IuvxpMndd0l6VNJZWY8lBWdIOrdxvftnkj5oZvdlO6R0uPuWxn8rkn4paWm2I0JWipRfUq4zjPyaBhRJbWosCLxH0nPuvirr8aTBzOaY2ezG92+S9GFJ/8x2VMlz9+vcfZG7L9H44yr+4O4rMh5W4szs8MYiXZnZ4ZI+Iim3d3ChtSLkl1SMDCO/pkfSWwDcL+mPkk40s81m9qUk++8SZ0i6SONV+zONr3OyHlTCFkh61Mz+pvFnXz3i7rm9vbQA5kl63Mz+KulJSb92999mPKauQ37lChmWH5nmFztuAwAABHC5DQAAIIAiCQAAIIAiCQAAIIAiCQAAIIAiCQAAIIAiCQAAIIAiCQAAIIAiCQAAIOB/Nvly+ZtiuQYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 720x576 with 4 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "from mlxtend.plotting import plot_decision_regions\n",
    "import matplotlib.gridspec as gridspec\n",
    "import itertools\n",
    "\n",
    "gs = gridspec.GridSpec(2, 2)\n",
    "\n",
    "fig = plt.figure(figsize=(10,8))\n",
    "\n",
    "labels = ['Logistic Regression', 'Random Forest', 'Naive Bayes', 'Ensemble']\n",
    "for clf, lab, grd in zip([clf1, clf2, clf3, eclf],\n",
    "                         labels,\n",
    "                         itertools.product([0, 1], repeat=2)):\n",
    "\n",
    "    clf.fit(X, y)\n",
    "    ax = plt.subplot(gs[grd[0], grd[1]])\n",
    "    fig = plot_decision_regions(X=X, y=y, clf=clf)\n",
    "    plt.title(lab)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 2 - Grid Search"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn import datasets\n",
    "\n",
    "iris = datasets.load_iris()\n",
    "X, y = iris.data[:, 1:3], iris.target"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.953 +/- 0.01 {'logisticregression__C': 1.0, 'randomforestclassifier__n_estimators': 20}\n",
      "0.960 +/- 0.01 {'logisticregression__C': 1.0, 'randomforestclassifier__n_estimators': 200}\n",
      "0.960 +/- 0.01 {'logisticregression__C': 100.0, 'randomforestclassifier__n_estimators': 20}\n",
      "0.960 +/- 0.01 {'logisticregression__C': 100.0, 'randomforestclassifier__n_estimators': 200}\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import GridSearchCV\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.naive_bayes import GaussianNB \n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "from mlxtend.classifier import EnsembleVoteClassifier\n",
    "\n",
    "clf1 = LogisticRegression(random_state=1)\n",
    "clf2 = RandomForestClassifier(random_state=1)\n",
    "clf3 = GaussianNB()\n",
    "eclf = EnsembleVoteClassifier(clfs=[clf1, clf2, clf3], voting='soft')\n",
    "\n",
    "params = {'logisticregression__C': [1.0, 100.0],\n",
    "          'randomforestclassifier__n_estimators': [20, 200],}\n",
    "\n",
    "grid = GridSearchCV(estimator=eclf, param_grid=params, cv=5)\n",
    "grid.fit(iris.data, iris.target)\n",
    "    \n",
    "cv_keys = ('mean_test_score', 'std_test_score', 'params')\n",
    "\n",
    "for r, _ in enumerate(grid.cv_results_['mean_test_score']):\n",
    "    print(\"%0.3f +/- %0.2f %r\"\n",
    "          % (grid.cv_results_[cv_keys[0]][r],\n",
    "             grid.cv_results_[cv_keys[1]][r] / 2.0,\n",
    "             grid.cv_results_[cv_keys[2]][r]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Note**: If the `EnsembleClassifier` is initialized with multiple similar estimator objects, the estimator names are modified with consecutive integer indices, for example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n"
     ]
    }
   ],
   "source": [
    "clf1 = LogisticRegression(random_state=1)\n",
    "clf2 = RandomForestClassifier(random_state=1)\n",
    "eclf = EnsembleVoteClassifier(clfs=[clf1, clf1, clf2], \n",
    "                              voting='soft')\n",
    "\n",
    "params = {'logisticregression-1__C': [1.0, 100.0],\n",
    "          'logisticregression-2__C': [1.0, 100.0],\n",
    "          'randomforestclassifier__n_estimators': [20, 200],}\n",
    "\n",
    "grid = GridSearchCV(estimator=eclf, param_grid=params, cv=5)\n",
    "grid = grid.fit(iris.data, iris.target)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Note**\n",
    "\n",
    "The `EnsembleVoteClass` also enables grid search over the `clfs` argument. However, due to the current implementation of `GridSearchCV` in scikit-learn, it is not possible to search over both, differenct classifiers and classifier parameters at the same time. For instance, while the following parameter dictionary works\n",
    "\n",
    "    params = {'randomforestclassifier__n_estimators': [1, 100],\n",
    "    'clfs': [(clf1, clf1, clf1), (clf2, clf3)]}\n",
    "    \n",
    "it will use the instance settings of `clf1`, `clf2`, and `clf3` and not overwrite it with the `'n_estimators'` settings from `'randomforestclassifier__n_estimators': [1, 100]`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 3 - Majority voting with classifiers trained on different feature subsets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Feature selection algorithms implemented in scikit-learn as well as the `SequentialFeatureSelector` implement a `transform` method that passes the reduced feature subset to the next item in a `Pipeline`.\n",
    "\n",
    "For example, the method\n",
    "\n",
    "    def transform(self, X):\n",
    "        return X[:, self.k_feature_idx_]\n",
    "        \n",
    "returns the best feature columns, `k_feature_idx_`, given a dataset X."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Thus, we simply need to construct a `Pipeline` consisting of the feature selector and the classifier in order to select different feature subsets for different algorithms. During `fitting`, the optimal feature subsets are automatically determined via the `GridSearchCV` object, and by calling `predict`, the fitted feature selector in the pipeline only passes these columns along, which resulted in the best performance for the respective classifier."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n",
      "/Users/kani/Documents/KatrinaLand/mlxtend/venv/lib/python3.7/site-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
      "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
      "\n",
      "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
      "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
      "Please also refer to the documentation for alternative solver options:\n",
      "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
      "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.947 +/- 0.02 {'pipeline__logreg__C': 1.0, 'pipeline__sfs__k_features': 1, 'randomforestclassifier__n_estimators': 20}\n",
      "0.947 +/- 0.02 {'pipeline__logreg__C': 1.0, 'pipeline__sfs__k_features': 1, 'randomforestclassifier__n_estimators': 200}\n",
      "0.947 +/- 0.02 {'pipeline__logreg__C': 1.0, 'pipeline__sfs__k_features': 2, 'randomforestclassifier__n_estimators': 20}\n",
      "0.960 +/- 0.01 {'pipeline__logreg__C': 1.0, 'pipeline__sfs__k_features': 2, 'randomforestclassifier__n_estimators': 200}\n",
      "0.953 +/- 0.01 {'pipeline__logreg__C': 1.0, 'pipeline__sfs__k_features': 3, 'randomforestclassifier__n_estimators': 20}\n",
      "0.960 +/- 0.01 {'pipeline__logreg__C': 1.0, 'pipeline__sfs__k_features': 3, 'randomforestclassifier__n_estimators': 200}\n",
      "0.953 +/- 0.01 {'pipeline__logreg__C': 100.0, 'pipeline__sfs__k_features': 1, 'randomforestclassifier__n_estimators': 20}\n",
      "0.953 +/- 0.01 {'pipeline__logreg__C': 100.0, 'pipeline__sfs__k_features': 1, 'randomforestclassifier__n_estimators': 200}\n",
      "0.960 +/- 0.01 {'pipeline__logreg__C': 100.0, 'pipeline__sfs__k_features': 2, 'randomforestclassifier__n_estimators': 20}\n",
      "0.960 +/- 0.01 {'pipeline__logreg__C': 100.0, 'pipeline__sfs__k_features': 2, 'randomforestclassifier__n_estimators': 200}\n",
      "0.960 +/- 0.01 {'pipeline__logreg__C': 100.0, 'pipeline__sfs__k_features': 3, 'randomforestclassifier__n_estimators': 20}\n",
      "0.960 +/- 0.01 {'pipeline__logreg__C': 100.0, 'pipeline__sfs__k_features': 3, 'randomforestclassifier__n_estimators': 200}\n"
     ]
    }
   ],
   "source": [
    "from sklearn import datasets\n",
    "\n",
    "iris = datasets.load_iris()\n",
    "X, y = iris.data[:, :], iris.target\n",
    "\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.naive_bayes import GaussianNB \n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "from mlxtend.classifier import EnsembleVoteClassifier\n",
    "from sklearn.pipeline import Pipeline\n",
    "from mlxtend.feature_selection import SequentialFeatureSelector\n",
    "\n",
    "clf1 = LogisticRegression(random_state=1)\n",
    "clf2 = RandomForestClassifier(random_state=1)\n",
    "clf3 = GaussianNB()\n",
    "\n",
    "# Creating a feature-selection-classifier pipeline\n",
    "\n",
    "sfs1 = SequentialFeatureSelector(clf1, \n",
    "                                 k_features=4,\n",
    "                                 forward=True, \n",
    "                                 floating=False, \n",
    "                                 scoring='accuracy',\n",
    "                                 verbose=0,\n",
    "                                 cv=0)\n",
    "\n",
    "clf1_pipe = Pipeline([('sfs', sfs1),\n",
    "                      ('logreg', clf1)])\n",
    "\n",
    "eclf = EnsembleVoteClassifier(clfs=[clf1_pipe, clf2, clf3], \n",
    "                              voting='soft')\n",
    "\n",
    "\n",
    "params = {'pipeline__sfs__k_features': [1, 2, 3],\n",
    "          'pipeline__logreg__C': [1.0, 100.0],\n",
    "          'randomforestclassifier__n_estimators': [20, 200]}\n",
    "\n",
    "grid = GridSearchCV(estimator=eclf, param_grid=params, cv=5)\n",
    "grid.fit(iris.data, iris.target)\n",
    "\n",
    "\n",
    "\n",
    "cv_keys = ('mean_test_score', 'std_test_score', 'params')\n",
    "\n",
    "for r, _ in enumerate(grid.cv_results_['mean_test_score']):\n",
    "    print(\"%0.3f +/- %0.2f %r\"\n",
    "          % (grid.cv_results_[cv_keys[0]][r],\n",
    "             grid.cv_results_[cv_keys[1]][r] / 2.0,\n",
    "             grid.cv_results_[cv_keys[2]][r]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The best parameters determined via GridSearch are:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'pipeline__logreg__C': 1.0,\n",
       " 'pipeline__sfs__k_features': 2,\n",
       " 'randomforestclassifier__n_estimators': 200}"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "grid.best_params_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, we assign these parameters to the ensemble voting classifier, fit the models on the complete training set, and perform a prediction on 3 samples from the Iris dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 1, 2])"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "eclf = eclf.set_params(**grid.best_params_)\n",
    "eclf.fit(X, y).predict(X[[1, 51, 149]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Manual Approach"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Alternatively, we can select different columns \"manually\" using the `ColumnSelector` object. In this example, we select only the first (sepal length) and third (petal length) column for the logistic regression classifier (`clf1`)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 1, 2])"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from mlxtend.feature_selection import ColumnSelector\n",
    "\n",
    "\n",
    "col_sel = ColumnSelector(cols=[0, 2])\n",
    "\n",
    "clf1_pipe = Pipeline([('sel', col_sel),\n",
    "                      ('logreg', clf1)])\n",
    "\n",
    "eclf = EnsembleVoteClassifier(clfs=[clf1_pipe, clf2, clf3],\n",
    "                              voting='soft')\n",
    "eclf.fit(X, y).predict(X[[1, 51, 149]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Furthermore, we can fit the `SequentialFeatureSelector` separately, outside the grid search hyperparameter optimization pipeline. Here, we determine the best features first, and then we construct a pipeline using these \"fixed,\" best features as seed for the `ColumnSelector`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.\n",
      "[Parallel(n_jobs=1)]: Done   4 out of   4 | elapsed:    0.0s finished\n",
      "Features: 1/2[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.\n",
      "[Parallel(n_jobs=1)]: Done   3 out of   3 | elapsed:    0.0s finished\n",
      "Features: 2/2"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best features (2, 3)\n"
     ]
    }
   ],
   "source": [
    "sfs1 = SequentialFeatureSelector(clf1, \n",
    "                                 k_features=2,\n",
    "                                 forward=True, \n",
    "                                 floating=False, \n",
    "                                 scoring='accuracy',\n",
    "                                 verbose=1,\n",
    "                                 cv=0)\n",
    "\n",
    "sfs1.fit(X, y)\n",
    "\n",
    "print('Best features', sfs1.k_feature_idx_)\n",
    "\n",
    "col_sel = ColumnSelector(cols=sfs1.k_feature_idx_)\n",
    "\n",
    "clf1_pipe = Pipeline([('sel', col_sel),\n",
    "                      ('logreg', clf1)])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 1, 2])"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "eclf = EnsembleVoteClassifier(clfs=[clf1_pipe, clf2, clf3], \n",
    "                              voting='soft')\n",
    "eclf.fit(X, y).predict(X[[1, 51, 149]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 5 - Using Pre-fitted Classifiers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn import datasets\n",
    "\n",
    "iris = datasets.load_iris()\n",
    "X, y = iris.data[:, 1:3], iris.target"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Assume that we previously fitted our classifiers:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn import model_selection\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.naive_bayes import GaussianNB \n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "import numpy as np\n",
    "\n",
    "clf1 = LogisticRegression(random_state=1)\n",
    "clf2 = RandomForestClassifier(random_state=1)\n",
    "clf3 = GaussianNB()\n",
    "\n",
    "for clf in (clf1, clf2, clf3):\n",
    "    clf.fit(X, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "By setting `fit_base_estimators=False`, it will enforce `use_clones` to be False and the `EnsembleVoteClassifier` will not re-fit these classifers to save computational time:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning: enforce use_clones to be False\n",
      "accuracy: 0.96\n"
     ]
    }
   ],
   "source": [
    "from mlxtend.classifier import EnsembleVoteClassifier\n",
    "import copy\n",
    "eclf = EnsembleVoteClassifier(clfs=[clf1, clf2, clf3], weights=[1,1,1], fit_base_estimators=False)\n",
    "\n",
    "labels = ['Logistic Regression', 'Random Forest', 'Naive Bayes', 'Ensemble']\n",
    "\n",
    "eclf.fit(X, y)\n",
    "\n",
    "print('accuracy:', np.mean(y == eclf.predict(X)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "However, please note that `fit_base_estimators=False` is incompatible to any form of cross-validation that is done in e.g., `model_selection.cross_val_score` or `model_selection.GridSearchCV`, etc., since it would require the classifiers to be refit to the training folds. Thus, only use `fit_base_estimators=False` if you want to make a prediction directly without cross-validation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 6 - Ensembles of Classifiers that Operate on Different Feature Subsets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If desired, the different classifiers can be fit to different subsets of features in the training dataset. The following example illustrates how this can be done on a technical level using scikit-learn pipelines and the `ColumnSelector`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "EnsembleVoteClassifier(clfs=[Pipeline(memory=None,\n",
       "                                      steps=[('columnselector',\n",
       "                                              ColumnSelector(cols=(0, 2),\n",
       "                                                             drop_axis=False)),\n",
       "                                             ('logisticregression',\n",
       "                                              LogisticRegression(C=1.0,\n",
       "                                                                 class_weight=None,\n",
       "                                                                 dual=False,\n",
       "                                                                 fit_intercept=True,\n",
       "                                                                 intercept_scaling=1,\n",
       "                                                                 l1_ratio=None,\n",
       "                                                                 max_iter=100,\n",
       "                                                                 multi_class='auto',\n",
       "                                                                 n_jobs=None,\n",
       "                                                                 penalty='l2',\n",
       "                                                                 random_state=None,\n",
       "                                                                 solver='lbfgs',\n",
       "                                                                 tol=0.0001,\n",
       "                                                                 v...\n",
       "                                             ('logisticregression',\n",
       "                                              LogisticRegression(C=1.0,\n",
       "                                                                 class_weight=None,\n",
       "                                                                 dual=False,\n",
       "                                                                 fit_intercept=True,\n",
       "                                                                 intercept_scaling=1,\n",
       "                                                                 l1_ratio=None,\n",
       "                                                                 max_iter=100,\n",
       "                                                                 multi_class='auto',\n",
       "                                                                 n_jobs=None,\n",
       "                                                                 penalty='l2',\n",
       "                                                                 random_state=None,\n",
       "                                                                 solver='lbfgs',\n",
       "                                                                 tol=0.0001,\n",
       "                                                                 verbose=0,\n",
       "                                                                 warm_start=False))],\n",
       "                                      verbose=False)],\n",
       "                       fit_base_estimators=True, use_clones=True, verbose=0,\n",
       "                       voting='hard', weights=None)"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.datasets import load_iris\n",
    "from mlxtend.classifier import EnsembleVoteClassifier\n",
    "from mlxtend.feature_selection import ColumnSelector\n",
    "from sklearn.pipeline import make_pipeline\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "\n",
    "iris = load_iris()\n",
    "X = iris.data\n",
    "y = iris.target\n",
    "\n",
    "pipe1 = make_pipeline(ColumnSelector(cols=(0, 2)),\n",
    "                      LogisticRegression())\n",
    "pipe2 = make_pipeline(ColumnSelector(cols=(1, 2, 3)),\n",
    "                      LogisticRegression())\n",
    "\n",
    "eclf = EnsembleVoteClassifier(clfs=[pipe1, pipe2])\n",
    "\n",
    "eclf.fit(X, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example 7 - A Note about Scikit-Learn SVMs and Soft Voting"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This section provides some additional technical insights in how probabilities are used when `voting='soft'`. \n",
    "\n",
    "\n",
    "Note that scikit-learn estimates the probabilities for SVMs (more info here: http://scikit-learn.org/stable/modules/svm.html#scores-probabilities) in a way that these may not be consistent with the class labels that the SVM predicts. This is an extreme example, but let's say we have a dataset with 3 class labels, 0, 1, and 2. For a given training example, the SVM classifier may predict class 2. However, the class-membership probabilities may look as follows:\n",
    "\n",
    "- class 0: 99%\n",
    "- class 1: 0.5%\n",
    "- class 2: 0.5%\n",
    "\n",
    "A practical example of this scenario is shown below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning: enforce use_clones to be False\n",
      "============\n",
      "Probas from SVM            : [0.00910384 0.47821605 0.51268012]\n",
      "Class from SVM             : 1\n",
      "Probas from SVM in Ensemble: [0.00910384 0.47821605 0.51268012]\n",
      "Class from SVM in Ensemble : 2\n",
      "============\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "from mlxtend.classifier import EnsembleVoteClassifier\n",
    "from sklearn.svm import SVC\n",
    "from sklearn.datasets import load_iris\n",
    "\n",
    "iris = load_iris()\n",
    "X, y = iris.data, iris.target\n",
    "\n",
    "clf2 = SVC(probability=True, random_state=4)\n",
    "clf2.fit(X, y)\n",
    "eclf = EnsembleVoteClassifier(clfs=[clf2], voting='soft', fit_base_estimators=False)\n",
    "eclf.fit(X, y)\n",
    "\n",
    "for svm_class, e_class, svm_prob, e_prob, in zip(clf2.predict(X),\n",
    "                                                 eclf.predict(X),\n",
    "                                                 clf2.predict_proba(X),\n",
    "                                                 eclf.predict_proba(X)):\n",
    "    if svm_class != e_class:\n",
    "        print('============')\n",
    "        print('Probas from SVM            :', svm_prob)\n",
    "        print('Class from SVM             :', svm_class)\n",
    "        print('Probas from SVM in Ensemble:', e_prob)\n",
    "        print('Class from SVM in Ensemble :', e_class)\n",
    "        print('============')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Based on the probabilities, we would expect the SVM to predict class 2, because it has the highest probability. Since the `EnsembleVoteClassifier` uses the `argmax` function internally if `voting='soft'`, it would indeed predict class 2 in this case even if the ensemble consists of only one SVM model.\n",
    "\n",
    "Note that in practice, this minor technical detail does not need to concern you, but it is useful to keep it in mind in case you are wondering about results from a 1-model SVM ensemble compared to that SVM alone -- this is not a bug."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# API"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "ename": "FileNotFoundError",
     "evalue": "[Errno 2] No such file or directory: '../../api_modules/mlxtend.classifier/EnsembleVoteClassifier.md'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mFileNotFoundError\u001b[0m                         Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-21-ead21bf47c9d>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0;32mwith\u001b[0m \u001b[0mopen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'../../api_modules/mlxtend.classifier/EnsembleVoteClassifier.md'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m'r'\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0mf\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      2\u001b[0m     \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mf\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mread\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mFileNotFoundError\u001b[0m: [Errno 2] No such file or directory: '../../api_modules/mlxtend.classifier/EnsembleVoteClassifier.md'"
     ]
    }
   ],
   "source": [
    "with open('../../api_modules/mlxtend.classifier/EnsembleVoteClassifier.md', 'r') as f:\n",
    "    print(f.read())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
